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Executive  Summary 


Introduction/Purpose  of  the  Study 

This  report  provides  the  Centers  for  Medicare  and  Medicaid  Services  (CMS)  with  an  overview  of 
the  questionnaire  development  portion  of  RTFs  Questionnaire  Development  and  Cognitive  Testing  Using 
Item  Response  Theory  (IRT)  project.  The  purpose  of  this  project  is  design  a  pool  of  survey  questions  to 
measure  beneficiary  knowledge  of  the  Medicare  program  in  future  rounds  of  the  Medicare  Current 
Beneficiary  Survey  (MCBS).  The  overarching  objective  of  the  project  is  to  assist  CMS  in  assessing  how 
well  the  National  Medicare  Education  Program  is  meeting  its  consumer  information  goals. 

In  the  second  phase  of  the  project,  RTI  is  to  assign  a  common  metric  to  knowledge  item  sets  (e.g., 
four  to  eight  individual  knowledge  questions)  through  the  use  of  Item  Response  Theory  (IRT).  IRT  uses 
a  statistical  model  to  express  the  relationship  between  an  individual's  response  to  an  item  and  the 
underlying  latent  variable  (e.g.,  knowledge  of  the  Medicare  program).  IRT  scores  are  based  on  both  the 
individual's  responses  and  the  characteristics  of  the  items  being  tested.  Accordingly,  IRT  scores  provide 
a  more  precise  estimate  of  true  ability  than  do  the  summed  scores  used  in  classical  test  theory.  An 
additional  benefit  of  using  IRT  is  that  it  is  possible  to  produce  scores  that  are  comparable  even  when 
respondents  receive  different  sets  of  items.  Therefore,  using  IRT  for  the  MCBS  knowledge  scale  is 
desirable  for  two  important  reasons:  1)  it  allows  researchers  to  adapt  the  scale  to  new  CMS  policies  or 
priorities,  as  well  as  new  Medicare  configurations  and  benefits,  by  changing  some  items;  and  2)  it  allows 
a  more  precise  measure  of  beneficiary  knowledge  by  preventing  memorization  of  items.  Ultimately, 
survey  items  can  replace  each  other,  allowing  knowledge  to  be  measured  consistently  over  time  while 
retaining  the  necessary  flexibility. 

Question  Development  Activities 

Several  key  principles  guided  the  development  of  knowledge  items.  An  effective  survey  item 
must  be  statistically  reliable  and  valid  and  should  demonstrate  both  content-  and  item-level  validity.  The 
item  must  have  only  one  correct  answer,  and  the  information  needed  to  answer  the  question  should  be 
widely  available  to  beneficiaries.  The  item  should  be  sufficiently  difficult  to  effectively  discriminate 
between  people  with  a  high  level  of  knowledge  and  people  with  a  lower  level  of  knowledge. 

To  meet  these  criteria,  we  developed  questionnaire  items  using  a  comprehensive  multistep 
process  that  included  (i)  background  research;  (ii)  review  of  existing  Medicare  informational  materials 
and  Medicare  knowledge  surveys,  including  the  MCBS;  and  (iii)  multiple  meetings  and  discussions  with 
CMS  and  the  project's  seven-member  Technical  Advisory  Panel  (TAP). 


RTI 


1 


Round  1  Cognitive  Testing 

We  tested  46  knowledge  questions  and  two  self-reported  knowledge  questions  between  June  28 
and  July  2,  2001.  Knowledge  scores  calculated  after  the  first  round  of  interviews  showed  that 
respondents  answered  between  15  and  43  of  51  knowledge  items  correctly,  or  29  to  84  percent,  with  a 
mean  score  of  61  percent.  After  all  interviews  were  completed,  project  staff  members  met  to  discuss  both 
general  and  specific  findings  that  emerged  from  Round  1  of  cognitive  testing.  We  first  addressed  issues 
that  were  relevant  to  all  the  question  items  (e.g.,  how  do  people  respond  to  the  way  in  which  the  questions 
are  asked,  how  well  do  they  understand  the  survey  instructions,  and  do  they  understand  terms  that  are 
used  throughout  the  questions?).  Then  we  addressed  issues  relating  to  individual  survey  items.  Based 
upon  the  results  of  the  first  round  of  testing  and  upon  feedback  from  the  TAP  and  CMS,  we  revised  the 
survey  questions  for  a  second  round  of  testing.  In  addition  to  the  specific  changes  indicated  by  testing, 
we  attempted  to  simplify  the  language  used  in  all  questions  and  to  use  the  same  terminology  wherever 
possible.  We  also  organized  the  questions  into  content  categories  and  added  introductory  statements  for 
each  group  of  questions  to  improve  comprehensibility. 

Round  2  Cognitive  Testing 

Round  2  of  cognitive  testing  was  conducted  between  August  23  and  August  28,  2001.  In  addition 
to  the  knowledge  items  that  we  tested  in  Round  1,  we  included  several  other  types  of  questions  for  testing 
in  Round  2.  These  items  included  self-reported  knowledge  items;  health  literacy  items;  questions 
focusing  on  beneficiaries'  reading  and  memory  skills  as  a  global  measure  of  their  cognitive  abilities;  and 
non-knowledge  questions  that  will  generate  data  on  beneficiaries'  information-seeking  attitudes  and 
behavior,  use  of  Medicare-related  information  and  decision-support  tools,  and  decision-making  habits. 
These  items  may  reveal  reasons  for  differences  in  beneficiaries'  knowledge  scores,  particularly  in  regard 
to  questions  about  their  exposure  to  informational  resources  including  the  Medicare  &  You  handbook. 
This  type  of  information  could  illuminate  how  existing  informational  resources  can  be  modified  to 
increase  beneficiary  knowledge. 

In  Round  2,  we  tested  53  knowledge  items  and  28  health  literacy  items.  For  the  knowledge  items, 
respondents  correctly  answered  between  23  and  41  of  the  53  items,  or  43  to  77  percent,  with  a  mean  score 
of  58  percent.  For  the  health  literacy  items,  respondents  correctly  answered  between  18  and  27  of  the 
28  items,  or  68  to  96  percent,  with  a  mean  score  of  79  percent.  A  handful  of  general  issues  emerged  as  a 
result  of  the  second  round  of  testing.  We  also  revisited  some  issues  studied  in  Round  1.  In  general,  fewer 
changes  were  suggested  by  the  second  round  testing  than  by  the  first,  suggesting  that  the  instrument 
worked  better  overall.  Respondents  seemed  to  understand  most  items  and  their  underlying  concepts  and 
had  little  trouble  with  the  questions  and  response  options. 
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Conclusions/Recommendations 

Based  on  the  findings  from  two  rounds  of  cognitive  testing,  we  designed  a  survey  pool  of 
99  questions.  The  questions  are  broken  down  into  the  following  ten  categories: 

•  Self-Reported  Knowledge  (n  =  3); 

•  Eligibility  for  and  Structure  of  Original  Medicare  (n  =  14); 

•  Medicare  +  Choice  (n  =  12); 

•  Plan  Choice  and  Health  Plan  Decision-Making  (n  =  7); 

•  Information  and  Assistance,  Beneficiary  Rights,  and  Quality  of  Care  (n  =  10); 

•  Medigap/Employer-Sponsored  Supplemental  Insurance  (n  =  4); 

•  Health  Literacy  Terminology  (n  =  1 1); 

•  Health  Literacy  Scenarios  (n  =  16); 

•  Cognitive  Abilities  (n  =  6);  and 

•  Non-Knowledge  Items  (n  =  16). 

We  recommend  that  the  next  step  in  the  development  of  the  MCBS  knowledge  items  be  a  pretest 
in  which  all  of  the  newly  developed  demonstrated  knowledge  items  are  administered  to  a  large  sample  of 
respondents  (i.e.,  at  least  1,000).  The  respondents  selected  for  the  pretest  should  be  representative  of  the 
population  to  whom  the  items  will  eventually  be  administered.  Proceeding  in  this  way  will  allow  us  to 
capitalize  on  the  potential  benefits  of  IRT.  Specifically,  using  IRT  will  produce  a  more  precise  measure 
of  beneficiary  knowledge,  allowing  CMS  to  more  accurately  determine  areas  in  which  beneficiaries  lack 
knowledge  and  could  use  more  information.  In  addition,  an  IRT-based  knowledge  scale  will  allow  CMS 
to  track  knowledge  over  time,  thereby  allowing  for  the  evaluation  of  the  effectiveness  of  interventions  and 
education  programs. 
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1.0  Introduction 


1 .1    OVERVIEW/PURPOSE  OF  THE  STUDY 

This  report  provides  the  Centers  for  Medicare  and  Medicaid  Services  (CMS)  with  an  overview  of 
the  questionnaire  development  portion  of  RTFs  Questionnaire  Development  and  Cognitive  Testing  Using 
Item  Response  Theory  (IRT)  project.  The  purpose  of  this  project  is  design  a  pool  of  survey  questions  to 
measure  beneficiary  knowledge  of  the  Medicare  program  in  future  rounds  of  the  Medicare  Current 
Beneficiary  Survey  (MCBS).  The  MCBS  is  a  recurring  multipurpose  survey  of  a  representative  sample  of 
the  Medicare  population;  therefore,  it  is  an  ideal  vehicle  with  which  to  examine  changes  in  knowledge 
and  behavior  over  time. 

The  overarching  objective  of  the  project  is  to  assist  CMS  in  assessing  how  well  the  National 
Medicare  Education  Program  (NMEP)  is  meeting  its  consumer  information  goals.  The  NMEP  is  intended 
to  increase  beneficiary  access  to,  awareness  of,  understanding  about,  and  use  of  the  information  to  make 
appropriate  health  plan  and  health  care  delivery  choices.  The  information  dissemination  elements  of  the 
NMEP  include  the  updated  Medicare  &  You  handbook,  the  www.medicare.gov  Internet  web  site,  the 
national  toll-free  help  line  l-800-MEDICAR(E),  and  a  variety  of  activities  intended  to  train  counselors  to 
help  Medicare  beneficiaries  make  informed  decisions  about  how  to  use  and  receive  their  Medicare 
benefits. 

We  designed  survey  questions  to  address  the  following  research  questions  that  will  measure  how 
well  NMEP  is  meeting  its  goals: 

♦  Are  beneficiaries  knowledgeable  about  Medicare  and  related  health  plan  options?  How  is 
their  knowledge  changing  over  time? 

♦  What  are  beneficiaries'  impressions  of  the  informational  resources  available  to  them? 

♦  Are  beneficiaries  using  the  informational  resources  available  to  them? 

♦  How  is  the  information  affecting  beneficiaries'  attitudes  and  decision-making  behavior? 

In  the  second  phase  of  the  project,  RTI  is  to  assign  a  common  metric  to  knowledge  item  sets  (e.g., 
four  to  eight  individual  knowledge  questions)  through  the  use  of  Item  Response  Theory  (IRT)  (Bann, 
2001).  IRT  uses  a  statistical  model  to  express  the  relationship  between  an  individual's  response  to  an 
item  and  the  underlying  latent  variable  (e.g.,  knowledge  of  the  Medicare  program).  IRT  scores  are  based 
on  both  the  individual's  responses  and  the  characteristics  of  the  items  being  tested.  Accordingly,  IRT 
scores  provide  a  more  precise  estimate  of  true  ability  than  do  the  summed  scores  used  in  classical  test 
theory.  An  additional  benefit  of  using  IRT  is  that  it  is  possible  to  produce  scores  that  are  comparable 
even  when  respondents  receive  different  sets  of  items.  Therefore,  using  IRT  for  the  MCBS  knowledge 
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scale  is  desirable  for  two  important  reasons:  1)  it  allows  researchers  to  adapt  the  scale  to  new  CMS 
policies  or  priorities,  as  well  as  new  Medicare  configurations  and  benefits,  by  changing  some  items;  and 
2)  it  allows  a  more  precise  measure  of  beneficiary  knowledge  by  preventing  memorization  of  items. 
Ultimately,  survey  items  can  replace  each  other,  allowing  knowledge  to  be  measured  consistently  over 
time  while  retaining  the  necessary  flexibility. 

Although  the  primary  purpose  is  to  develop  survey  items  to  measure  beneficiary  knowledge,  we, 
felt  that  it  was  important  to  include  a  select  set  of  non-knowledge  items.  These  items  enable  researchers 
to  fully  evaluate  whether  the  NMEP  is  working  as  intended  by  assessing  beneficiaries'  impressions, 
knowledge,  and  use  of  available  informational  resources. 

The  remainder  of  this  chapter  summarizes  the  item  development  activities  that  we  conducted, 
including  the  development  of  key  content  areas  that  guided  the  subsequent  development  of  key 
knowledge  concepts  and  survey  items.  Chapters  2  and  3  detail  the  protocols  and  results  of  each  of  two 
rounds  of  cognitive  testing  conducted  to  evaluate  the  survey  items.  Finally,  Chapter  4  presents 
recommendations  for  the  future  and  a  discussion  of  next  steps  for  the  project. 

1 .2    QUESTION  DEVELOPMENT  ACTIVITIES 

Several  key  principles  guided  the  development  of  knowledge  items.  An  effective  survey  item 
must  be  statistically  reliable  and  valid  and  should  demonstrate  both  content-  and  item-level  validity.  The 
item  must  have  only  one  correct  answer,  and  the  information  needed  to  answer  the  question  should  be 
widely  available  to  beneficiaries.  The  item  should  be  sufficiently  difficult  to  effectively  discriminate 
between  people  with  a  high  level  of  knowledge  and  people  with  a  lower  level  of  knowledge. 

To  meet  these  criteria,  we  developed  questionnaire  items  using  a  comprehensive  multistep 
process  that  included  (i)  background  research;  (ii)  review  of  existing  Medicare  informational  materials 
and  Medicare  knowledge  surveys,  including  the  MCBS;  and  (iii)  multiple  meetings  and  discussions  with 
CMS  and  the  project's  seven-member  Technical  Advisory  Panel  (TAP).1 

Background  Research 

We  began  item  development  efforts  by  conducting  background  research  to  identify  a  variety  of 
print  materials  and  Internet  web  sites  that  contained  relevant  and  accurate  information  about  the  Medicare 
program.  To  ensure  that  our  search  was  comprehensive,  we  consulted  with  CMS,  TAP  members,  and 
other  individuals  with  extensive  knowledge  of  the  Medicare  program.  Materials  that  we  identified 
included  informational  documents  provided  by  Medicare  to  beneficiaries  to  inform  them  about  the 
Medicare  program,  materials  from  consumer  advocacy  groups  and  nonprofit  organizations,  research 
studies  that  examined  beneficiaries'  knowledge  of  Medicare,  and  survey  questions  used  to  measure 
beneficiaries'  knowledge  of  Medicare. 


Please  see  Appendix  A  for  a  complete  list  of  TAP  members.  More  information  about  the  role  of  the 
TAP  is  provided  in  the  pages  that  follow. 
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In  our  initial  information-gathering  efforts,  we  attempted  to  identify  concepts  relevant  to  one  of 
six  key  knowledge  content  areas  (initially  presented  to  CMS  during  the  proposal  stage).  These 
knowledge  content  areas  are 

♦  Eligibility  for  and  Structure  of  Original  Medicare; 

♦  Key  Differences  between  Original  Medicare,  Supplemental  Insurance,  and  Managed  Care 
Options; 

♦  Differences  in  Quality  of  Care  Exist; 

♦  Beneficiaries  Have  Rights  and  Protections; 

♦  How  to  Get  More  Information/Assistance;  and 

♦  Special  Populations. 

Two  additional  non-knowledge  content  areas  were 

♦  Feedback  on  and  Use  of  Information  and 

♦  Impact  of  the  NMEP  on  Attitudes  and  Decision-making  Behavior. 

We  also  solicited  input  from  the  TAP  about  the  content  areas.  We  asked  the  members  to 
recommend  the  primary  conceptual  areas  about  which  Medicare  beneficiaries  should  be  knowledgeable. 
As  a  result  of  this  effort  and  our  review  of  background  materials,  we  revised  and  expanded  our  draft 
knowledge  content  areas  to  the  following  eleven  areas: 

♦  Eligibility  for  and  Structure  of  Original  Medicare  (including  covered  benefits  and  out-of- 
pocket  costs); 

♦  Key  Differences  between  Original  Medicare,  Supplemental  Insurance,  and  Managed  Care 
Options; 

♦  Differences  in  Quality  of  Care  Exist; 

♦  Beneficiaries  Have  Rights  and  Protections; 

♦  How  to  Get  More  Information/Assistance; 

♦  Self-Reported  Knowledge  (new); 

♦  Special  Populations; 

♦  Health  Literacy  (new); 

♦  Cognitive  Abilities  (new); 
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♦  Feedback  on  the  Use  of  Information  and  Impact  of  NMEP  on  Attitudes  and  Decision-Making 
Behavior;  and 

♦  Other  (for  concepts  that  did  not  fit  easily  into  the  above  content  areas) 

Development  of  the  Master  Document 

The  next  step  was  to  organize  all  of  the  information  that  we  collected  in  our  background  review 
into  a  single  document  ("the  Master  Document").  This  Master  Document  was  designed  to  easily  sort  key 
concepts  into  their  relevant  knowledge  and  non-knowledge  content  areas  and  to  match  concepts  with 
sample  survey  questions.  For  each  content  area,  we  listed  critical  concepts  pertaining  to  the  content  area, 
the  sources  of  these  concepts,  example  questions  relevant  to  the  concepts,  and  the  sources  of  the  example 
questions.  An  additional  column  for  comments  allowed  us  to  include  notes  for  consideration  by  other 
project  team  members. 

We  continually  added  to,  updated,  and  refined  the  Master  Document  as  a  working  tool.  Each 
knowledge  concept  and  question  included  in  the  Master  Document  was  reviewed  systematically,  and  any 
missing  information  was  completed  or  developed.  For  example,  for  concepts  that  did  not  have  an 
accompanying  example  question,  we  first  searched  existing  surveys  for  a  survey  item  that  would  address 
the  concept.  If  no  appropriate  example  item  was  found,  we  developed  a  new  survey  item.  Prior  to  the 
project  kickoff  meeting,  we  submitted  to  CMS  a  draft  of  the  Master  Document,  which  included  ten 
potential  content  areas  and  170  draft  questions. 

Project  Kickoff  Meeting 

On  June  6, 2001,  senior  RTI  project  staff  met  with  CMS  for  a  kickoff  meeting  to  discuss  the 
project  and  work  completed  to  date  and  to  obtain  guidance  regarding  our  revised  content  areas  and  draft 
concepts.  The  Master  Document  was  revised  based  on  comments  from  CMS  staff.  After  meeting,  we 
proceeded  with  the  question  development  process  as  intended. 

TAP  Meeting 

As  noted  earlier,  we  relied  on  the  expertise  of  a  seven-member  team  of  Medicare  experts 
recruited  to  serve  as  a  TAP  during  the  questionnaire  development  process.  The  members  of  this  panel 
have  multidisciplinary  expertise  in  areas  such  as  health  policy,  health  education,  consumer  information 
and  rights,  decision-making  research,  law,  and  cognitive  psychology.  TAP  members  were  involved  at 
various  stages  during  this  project,  developing  content  areas,  identifying  key  concepts,  reviewing 
questions,  reviewing  the  results  of  the  first  and  second  rounds  of  cognitive  testing,  and  recommending 
changes  to  question  items. 

We  convened  an  in-person  TAP  meeting  on  June  21,  2001,  at  the  RTI  headquarters  in  Research 
Triangle  Park,  NC.  Prior  to  the  meeting,  participants  were  asked  to  review  an  initial  draft  of  knowledge 
questions  compiled  by  project  staff.  The  draft  knowledge  questions  incorporated  feedback  given  by  CMS 
at  the  project  kickoff  meeting. 
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Participants  made  several  important  decisions  at  this  meeting.  First,  it  was  decided  that  the 
Medicare  &  You  handbook  would  serve  as  the  primary  (although  not  the  only)  guide  for  the  question 
development  process.  Second,  most  questions  would  have  no  more  than  three  response  options  because 
older  adults  have  limitations  with  working  memory  capacity,  particularly  when  a  survey  is  administered 
orally.  (The  MCBS  is  administered  using  computer-assisted  personal  interviewing  techniques.) 
Participants  also  agreed  that  the  three  response  options  would  be  in  addition  to  a  "Don't  know"  response 
for  each  question.  The  "Don't  know"  response  option  is  not  designed  to  be  read  aloud,  but  beneficiaries 
will  be  informed  at  the  beginning  of  the  string  of  knowledge  question  that  "Don't  know"  is  an  acceptable 
response.  Finally,  it  is  important  to  include  self-reported  knowledge  questions  both  at  the  beginning  and 
at  the  end  of  the  string  of  demonstrated  knowledge  items  to  evaluate  ordering  effects. 

After  much  discussion  and  consideration  of  the  draft  content  areas  and  questions,  participants 
decided  that  the  following  five  knowledge  content  areas  were  the  most  critical: 

♦  Eligibility  for  and  Structure  of  Original  Medicare, 

♦  Medicare  +  Choice, 

♦  Plan  Choices  and  Health  Plan  Decision-Making, 

♦  Information  and  Assistance  (broadly  defined  to  include  beneficiary  rights  and  quality  of 
care),  and 

♦  Medigap/Employer-sponsored  Supplemental  Insurance. 

In  addition  to  the  knowledge  content  areas,  we  recommended  the  inclusion  of  the  following  four 
non-knowledge  content  areas: 

♦  Self-Reported  Knowledge  (primarily  for  psychometric  purposes), 

♦  Health  Literacy  (to  evaluate  whether  health  literacy  plays  a  role  in  beneficiaries'  ability  to 
comprehend  Medicare  information), 

♦  Cognitive  Abilities,  and 

♦  Other  Non-Knowledge  Items  (for  analytic  purposes). 

The  non-knowledge  content  areas  enable  us  to  collect  information  that  might  reveal  reasons  for 
differences  in  beneficiaries'  knowledge  scores. 

After  the  TAP  meeting,  CMS  provided  the  following  guidance  regarding  further  question 
development: 

♦  CMS  agreed  with  the  decision  to  have  three  response  options  for  most  questions  and  a  "Don't 
know"  response  option  available  for  all  items. 

♦  CMS  indicated  a  preference  for  Yes/No  and  True/False  questions. 
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♦    CMS  specified  that  the  language  used  in  all  questions  should  be  direct  and  simple,  and  trick 
questions  should  be  avoided. 

Health  Literacy  Expert  and  Conference 

Since  health  literacy  may  play  an  important  role  in  beneficiaries'  ability  to  comprehend 
Medicare-related  information,  we  felt  that  it  was  important  to  collect  additional  information  about  health 
literacy  in  the  Medicare  population.  On  June  13,  2001,  two  project  staff  members  attended  Medicare 
Education  and  Health  Literacy:  Techniques  for  Educating  Older  Adults,  a  conference  sponsored  by  the 
Center  for  Medicare  Education.  Dr.  Rima  Rudd,  a  Harvard  professor  who  has  done  extensive  work  on 
health  literacy,  was  a  presenter  at  the  conference.  We  consulted  with  Dr.  Rudd  during  the  development  of 
health  literacy  questions  and  scenarios,  which  were  tested  during  the  second  round  of  cognitive  testing 
(for  a  more  detailed  discussion  of  these  questions,  please  refer  to  Chapter  3). 


RTI 


10 


2 


2.0  Cognitive  Testing:  Round  1 


2.1  INTRODUCTION 

Cognitive  testing  is  routinely  used  for  survey  questionnaire  development  (Forsyth  and  Lessler, 
1991;  DeMaio  and  Rothgeb,  1996).  We  use  cognitive  testing  regularly  to  determine  question 
comprehension  (What  do  respondents  think  the  question  is  asking?  What  do  specific  words  and  phrases 
in  the  question  mean  to  them?),  information  retrieval  (What  do  respondents  need  to  recall  to  answer  the 
question?  How  do  they  do  this?),  and  decision  processes  (How  do  respondents  choose  their  answers?). 
In  this  chapter  we  briefly  summarize  key  findings  from  the  first  round  (Round  1)  of  cognitive  testing 
interviews,  which  were  conducted  between  June  28  and  July  2,  2001. 

We  tested  a  total  of  48  questions  organized  into  the  following  nine  categories: 

♦  Self-Reported  Knowledge  (n  =  2)  (questions  1  and  48), 

♦  Eligibility  for  and  Structure  of  Original  Medicare  (n  =  13)  (questions  2  through  14), 

♦  Managed  Care  and  Medicare  +  Choice  Program  (n  =  9)  (questions  15  through  23), 

♦  Medigap-Specific  Issues  (n  =  1)  (question  24), 

♦  Similarities  and  Differences  Between  Plan  Choices  (n  =  8)  (questions  25  through  32), 

♦  Differences  in  Quality  of  Care  (n  =  2)  (questions  33  and  34), 

♦  Beneficiary  Rights  and  Protections  (n  =  2)  (questions  35  and  36), 

♦  Information  and  Assistance  (n  =  6)  (questions  37  through  42),  and 

♦  Special  Subgroup  Issues  (n  =  5)  (questions  43  through  47). 

In  some  cases,  several  questions  dealing  with  the  same  concept  were  tested.  This  overlap  will 
allow  different  questions  that  address  the  same  concept  to  be  used  on  different  versions  of  the 
questionnaire,  thereby  preventing  MCBS  respondents  from  being  asked  the  same  knowledge  questions 
every  year.  In  addition,  testing  similar  questions  at  this  early  stage  of  development  provided  us  with  more 
freedom  to  drop  any  questions  found  to  be  problematic. 
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2.2    RESPONDENT  CHARACTERISTICS 

During  the  first  round  of  cognitive  testing  we  interviewed  ten  respondents  with  varying 
characteristics.1  Participants  were  recruited  from  area  senior  centers  and  senior  housing  facilities  via 
flyers  posted  in  public  places  and  by  word  of  mouth.  Respondents  who  had  Medicare  and  had  not 
participated  in  a  health-related  study  during  the  past  six  months  qualified  for  the  cognitive  testing.  We 
recruited  27  eligible  adults.  We  chose  the  final  group  of  participants  to  reflect  diversity  in  education, 
health  insurance,  race,  age,  and  gender.  Table  1  provides  a  breakdown  of  key  characteristics  by 
respondent.  Table  2  summarizes  these  characteristics  across  all  respondents. 


Table  1 :  Sociodemographic  and  Health  Insurance  Characteristics  by  Respondent 


Respondent 

Age 

Medigap 

Medicare  HMO 

Medicaid 

Education 

Ethnicity 

1 

66 

Y 

N 

N 

Some  college 

Black 

2 

72 

N 

N 

Y 

Some  HS 

Black 

3 

68 

Y 

N 

N 

Some  HS 

White 

4 

77 

Y 

N 

N 

Graduated  HS 

White 

5 

61 

N 

N 

Y 

Less  than  HS 

White 

6 

69 

N 

N 

Y 

Graduated  HS 

Other 

7 

64 

N 

N 

N 

Some  college 

White 

8 

76 

N 

N 

N 

Graduated  college 

White 

9 

74 

N 

Y 

Y 

Some  HS 

Black 

10 

78 

Y(E) 

N 

N 

Graduated  college 

White 

Y  =  Yes,  N  =  No,  E  =  Employer-sponsored 


1    In  order  to  comply  with  OMB  requirements,  two  respondents  were  each  asked  only  half  of  the 
questions  from  the  questionnaire. 
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Table  2:  Sociodemographic  and  Health  Insurance  Characteristics 
By  Number  of  Respondents  (n  =  10) 


Number  of 

oncircici6ri5iic 

DocnAnHante 

ntfofjuiiuciiio 

Aae 

60-69 

5 

70-79 

5 

Education 

1  p^q  than  hinh  ^phool 

LCOO  il  lul  1   1  MUI  1  Owl  IvUI 

1 

1 

Qnmp  hinh  ^f*hnnl 

<3 
O 

firarii  i?itprl  hinh  ^phonl 

\Jil  uUUCll^U  1  MM'  •  Owl  IWI 

2 

Somp  nnllpne 

2 

Colleae  araduate  or  above 

2 

Ethnicity 

Black 

3 

Whitp 

V  V  1  1 1  Iv? 

R 
u 

Other 

1 

Insurance* 

Medicare  only 

2 

Medigap 

3 

Employer-sponsored  supplement 

1 

Medicare  HMO 

1 

Medicaid 

4 

*Totals  eleven  because  one  respondent  had  both  a  Medicare  HMO  and  Medicaid. 


2.3  RESULTS 

Knowledge  scores  calculated  after  the  first  round  of  interviews  showed  that  respondents  answered 
between  15  and  43  (out  of  51)  knowledge  items  correctly,  or  29  to  84  percent,  with  a  mean  score  of  61 
percent.   Individual  scores  are  presented  in  Table  3,  along  with  respondents'  education  level.  After  all 
interviews  were  completed,  project  staff  members  met  to  discuss  both  general  and  specific  findings  that 
emerged  from  Round  1  of  cognitive  testing. 


Although  there  were  only  46  knowledge  questions,  several  of  the  questions  had  multiple  parts  (i.e., 
questions  embedded  within  the  question).  Therefore,  it  was  possible  to  receive  a  maximum  score  of 
51.  Scores  were  computed  for  all  but  the  two  respondents  who  completed  half  interviews. 
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Table  3:  Interviewee  Scores  by  Education  Level 


Education 

Number  Correct/ 
Total  Number 

Percentage  Correct 

Less  than  High  School 

27/51 

53% 

Some  High  School 

15/25 

60% 

Some  High  School 

43/51 

84% 

Some  High  School 

15/51 

29% 

Graduated  High  School 

28/51 

55% 

Graduated  High  School 

17/25 

68% 

Some  College 

41/51 

80% 

Some  College 

31/51 

61% 

Graduated  College 

35/51 

69% 

Graduated  College 

30/51 

59% 

Mean  Score 

31.25* 

61%* 

*  Mean  score  calculated  for  only  those  respondents  that  completed  the  entire  survey  (out  of  51). 


2.3.1  General  Issues/Findings 

We  first  addressed  issues  that  were  relevant  to  all  the  question  items  (e.g.,  how  do  people  respond 
to  the  way  in  which  the  questions  are  asked,  how  well  do  they  understand  the  survey  instructions,  and  do 
they  understand  terms  that  are  used  throughout  the  questions?).  General  findings  from  Round  1  of 
cognitive  testing  include  the  following: 

1.   We  found  that  all  of  the  respondents  used  the  "Don't  know"  answer  option  several  times 
throughout  the  questionnaire.  Because  "Don't  know"  was  not  read  as  an  answer  option  for 
each  question,  interviewers  reminded  respondents  that  "Don't  know"  was  an  option  as 
necessary.  At  least  a  couple  of  respondents  asked  to  change  an  answer  to  a  previous  question 
when  reminded  of  the  "Don't  know"  option,  suggesting  that  respondents  may  have  forgotten. 
When  later  probed  further  about  this  issue,  most  respondents  reported  that  rather  than  guess 
they  had  said  "Don't  know"  when  they  did  not  know  the  answer.  These  experiences  suggest 
the  importance  of  retaining  the  "Don't  know"  option. 


2.  Interview  results  show  that  changes  in  question  format  (e.g.,  Yes/No,  True/False,  and 
multiple  choice)  did  not  present  problems  for  respondents.  In  addition,  respondents  did  not 
have  any  difficulty  with  survey  items  that  contained  four  response  options  (i.e.,  those  that 
included  two  answers  plus  "Both"  and  "Neither"). 

3.  Testing  showed  that  it  was  difficult  for  Medicare  beneficiaries  who  qualify  for  Medicaid  to 
separate  the  services  covered  by  each  program  when  answering  the  questions.  That  is,  dual- 
eligible  respondents  based  their  answers  about  coverage  of  certain  items  on  whether  the 
service  is  covered  for  them  personally,  regardless  of  which  public  insurance  program — 
Medicare  or  Medicaid — actually  covered  the  services. 


RTI 


14 


RTI 


Cognitive  Testing:  Round  1 


4.  We  also  investigated  respondents'  comprehension  of  several  critical  terms:  the  Original 
Medicare  plan,  managed  care,  Medicare  +  Choice,  and  other  phrases  used  to  describe  the 
health  plan  choices  available  to  Medicare  beneficiaries.  We  found  that  the  term  the  Original 
Medicare  plan  was  difficult  for  most  respondents,  in  some  cases  even  after  a  definition  was 
given.  Several  respondents  thought  that  "Original"  suggested  that  "it  may  have  changed"  or 
it  "was  the  first  version."  (One  respondent  gave  the  example  of  Coca-Cola®  advertising  'The 
Original  Coca  Cola.")  As  a  result,  respondents'  ability  to  provide  accurate  answers  to  survey 
questions  that  contained  this  term  was  compromised  by  varying  degrees.  For  example,  one 
respondent  first  said  that  she  could  not  answer  any  of  the  questions  that  asked  about  "Original 
Medicare"  because  she  "didn't  know  what  Medicare  used  to  be  like  when  it  first  came  out." 
Another  respondent  noted  the  "poor  wording  choice"  but  was  able  to  comprehend  and  answer 
the  questions  appropriately. 

The  term  managed  care  was  problematic  for  several  respondents  and  most  respondents  had 
not  heard  of  the  term  Medicare  +  Choice.  Although  some  had  general  ideas  of  what  these 
terms  meant,  other  respondents  were  completely  unfamiliar  with  them  or  offered  incorrect 
definitions.  For  example,  one  respondent  thought  that  managed  care  referred  to  "a  case 
worker-type  person  who  coordinates  health  care."  Another  thought  that  it  referred  to 
"someone  who  comes  into  your  home  to  take  care  of  you,"  while  a  third  said  "there  are  live- 
in  places  called  managed  care."  However,  further  probing  suggested  that  confusion  over 
these  questions  resulted  from  respondents'  lack  of  knowledge  about  managed  care  rather  than 
from  difficulties  with  the  questions  themselves.  We  chose  to  test  terminology  more 
extensively  during  Round  2  of  cognitive  testing. 

5.  We  also  explored  different  terms  that  could  be  used  to  describe  the  various  Medicare  health 
plan  options  in  an  effort  to  find  one  phrase  that  could  be  used  to  refer  collectively  to  the 
Original  Medicare  plan,  the  Original  Medicare  plan  with  a  supplement,  and  Medicare  + 
Choice  options.  We  specifically  probed  respondents  to  determine  whether  they  had  a 
preference  for  any  of  the  following  terms:  health  insurance  plans,  health  plan  choices, 
health  plans,  Medicare  health  plans,  or  Medicare  insurance  options.  Unfortunately,  a 
consensus  was  not  reached.  Several  indicated  that  having  the  term  insurance  in  the  phrase 
was  helpful  in  conveying  the  intended  meaning,  as  were  the  terms  option  and  choices. 
Therefore,  we  modified  the  questions  to  use  these  terms  when  appropriate. 

6.  Respondents  indicated  no  clear  preference  between  the  phrase  "people  on  Medicare"  or 
"people  with  Medicare."  Therefore  we  decided  to  use  "people  with  Medicare"  to  be 
consistent  with  the  Medicare  &  You  handbook  and  other  materials  prepared  by  CMS. 

2.3.2  Specific  Issues/Findings  by  Question 

In  the  following  section,  we  present  specific  findings  from  the  first  round  of  cognitive  testing  as 
they  relate  to  individual  survey  items.  Where  relevant,  we  include  the  modifications  we  recommended 
following  testing.  Please  note  that  question  numbers  refer  to  the  survey  items  as  numbered  in  the 
Round  1  cognitive  testing  protocol  (see  Appendix  B). 

Ql .      No  problems  were  noted. 

Q2.      One  respondent  who  had  Medicaid  was  not  sure  whether  Medicaid  or  Medicare  was  for  people 
with  low  or  moderate  incomes. 
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Q3.  Most  of  the  respondents  knew  that  Medicare  has  two  parts,  but  some  were  unsure  of  what  the 
parts  are  or  which  part  pays  for  physician  visits  and  which  pays  for  hospitalizations.  Some  of 
these  respondents  said  that  they  made  educated  guesses. 

Q4.      Although  most  respondents  knew  what  a  premium  is,  at  least  two  people  answered  the  question 
incorrectly  because  of  their  individual  situations.  One  respondent  did  not  pay  a  premium  because 
he  met  the  income  requirements  for  Medicaid.  The  other  respondent  was  unaware  that  a  premium 
was  deducted  monthly  from  his  social  security  check.  We  discovered  that,  although  most 
respondents  answered  correctly,  respondents  were  answering  based  on  their  own  situations  as 
opposed  to  thinking  about  "most  people  on  Medicare."  One  respondent  who  did  think  about 
"most  people  on  Medicare"  said  that  the  answer  should  be  "half  and  half  (i.e.,  true  and  false) 
because  she  does  not  pay  a  premium  but  knows  that  others  do. 

Q5.      No  problems  were  noted. 

Q6  and  Q7.  The  term  Original  Medicare  was  problematic  (please  see  the  discussion  in  Chapter  2.3.1). 

Q8.      None  of  the  four  respondents  with  Medicaid  coverage  were  able  to  distinguish  between  services 
covered  by  Medicare  and  those  covered  by  Medicaid.  For  example,  if  a  question  asked  about 
services  for  which  Medicare  pays,  respondents  answered  based  on  whether  the  benefit  was 
available  to  them  personally — through  either  Medicare  or  Medicaid — rather  than  if  it  was 
available  to  "most  people  on  Medicare."  Three  of  the  four  respondents  on  Medicaid  incorrectly 
answered  that  Original  Medicare  pays  for  "All  health  care  costs"  based  on  their  experience  of 
having  all  of  their  health  care  covered.  This  issue  of  respondents  answering  questions  based  on 
their  own  experiences  proved  problematic  throughout  the  questionnaire. 

Q9.      Two  male  respondents  noted  that  the  examples  of  preventive  screening  services  provided  in  the 
questions  pertained  only  to  women.  An  example  of  a  preventive  service  that  is  relevant  for  both 
men  and  women  (flu  shot)  was  added. 

Q10,  Q11,Q12.  Respondents  were  asked  questions  about  emergency  services  under  the  Original 

Medicare  plan  (Q10)  and  under  a  Medicare  HMO  (Ql  1).  Q12  combined  both  of  these  concepts 
into  one  question.  Respondents  were  asked  whether  they  preferred  Q10  and  Ql  1  or  Q12.  No 
clear  preference  emerged.  However,  we  decided  that  Q10  should  be  modified  to  address  specific 
comments  that  respondents  made  (e.g.,  they  were  unsure  whether  to  include  urgent  trips  to  the 
doctor,  ambulance  service,  etc.,  when  answering  Q10;  whereas  Q12  specifically  asks  about 
whether  a  visit  to  the  nearest  hospital  emergency  room  would  be  covered). 

Q13.     No  specific  problems  were  identified,  but  the  decision  was  made  to  test  the  phrase  health  care 
services  in  Round  2  to  determine  if  respondents  understand  the  term. 

Q14.     Several  respondents  said  they  were  unsure  how  to  answer  Q14d  (Does  Medicare  cover  care  in  a 
nursing  home?)  because  nursing  home  care  is  covered  under  certain  conditions  for  short  periods 
of  time.  This  issue  will  be  addressed  with  TAP  members  to  determine  an  appropriate  solution. 
Also,  because  these  services  are  covered  under  Medicaid,  respondents  with  Medicaid  answered 
that  most  of  these  services  are  covered.  (See  discussion  of  respondents  answering  questions 
based  on  their  own  experiences  in  Chapter  2.3.1.)  We  decided  to  replace  the  term  Original 
Medicare  with  Medicare  and  to  add  the  words  "whether  or  not"  and  "usually"  to  the  stem. 

Q15.     The  term  managed  care  confused  many  respondents.  (See  discussion  in  Chapter  2.3.1.) 
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Q16.     Most  respondents  had  not  heard  of  the  term  Medicare  +  Choice.  (See  discussion  in 

Chapter  2.3.1.)  A  simplified  version  of  this  question  was  developed  and  tested  in  Round  2. 

Q17.     This  question  is  similar  to  Q16.  Only  one  of  these  questions  will  be  kept.  No  change  was  made; 
the  question  was  investigated  further  in  Round  2. 

Q18.     For  the  most  part,  this  question  was  not  problematic,  but  one  respondent  with  Medicaid  said  that 
she  did  not  know  what  a  Medicare  HMO  is  or  what  it  covers. 

Q19.     No  problems  were  noted. 

Q20.     Answer  option  C  (Never)  will  be  changed  to  "At  the  end  of  the  year"  to  match  the  wording  of  the 
other  response  options.  Several  respondents  reported  discounting  the  response  option  right  away. 
This  makes  it  easier  for  respondents  to  guess  correctly  (between  answer  options  A  and  B)  if  they 
are  not  sure  of  the  correct  answer.  Also,  some  respondents  reported  difficulty  answering  this 
question  because  they  had  no  experience  with  HMOs  and  did  not  believe  that  this  question  was 
relevant  to  them. 

Q21.     No  problems  were  noted.  Answer  choice  C  (Never)  was  changed  to  "At  the  end  of  the  year"  to 
be  comparable  to  Q20. 

Q22.     This  question  was  clarified  by  adding  "if  one  is  available"  to  the  end  of  the  question.  The 

question  now  reads  "If  your  Medicare  HMO  stops  serving  people  with  Medicare  in  your  area,  can 
you  join  another  HMO  if  one  is  available?" 

Q23.  Many  respondents  reported  difficulty  answering  this  question  because  they  had  no  experience 
with  Medicare  HMOs.  The  relevance  of  this  question  was  examined  in  Round  2  of  cognitive 
testing. 

Q24.     "People  with  Medicare  coverage"  was  changed  to  "People  with  Medicare"  because  project  staff 
suspected  that  the  word  coverage  may  have  been  unclear. 

Q25.     "A  private  fee-for-service  plan"  will  be  dropped  as  an  answer  choice  and  replaced  with  "Both" 

and  "Neither."  In  addition,  a  separate  question  on  private  fee-for-service  plans  will  be  developed 
and  added  to  the  questionnaire  because  several  respondents  did  not  know  what  a  private  fee-for- 
service  plan  is.  Therefore,  we  believe  it  is  important  to  test  the  concept  separately.  Finally,  the 
word  through  was  changed  to  with. 

Q26.     No  problems  were  noted. 

Q27.     This  question  will  be  dropped  in  Round  2.  Almost  all  respondents  answered  this  question 

correctly.  This  suggests  that  the  item  was  too  easy  and  therefore  would  not  be  a  discriminating 
knowledge  question. 

Q28.     Respondents  were  read  a  number  of  alternate  phrases  that  could  be  used  to  replace  health 

insurance  options.  These  phrases  included  health  insurance  plans,  health  plan  choices,  health 
plans,  Medicare  health  plans,  and  Medicare  insurance  options.  Although  no  clear  preference 
emerged,  we  determined  that  using  choice  or  option  helped  convey  the  concept  of  choice.  In 
addition,  including  insurance  in  the  phrase  was  helpful,  while  including  plan  was  not.  Answer 
option  A  will  be  changed  to  "the  Original  Medicare  plan." 
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Q29.     The  answer  option  'They  are  both  about  the  same"  was  replaced  with  'They  will  both  pay  for 
prescription  drugs."  This  substitution  was  tested  in  Round  2. 

Q30.     Several  respondents  were  unsure  what  "other  Medicare  health  plans"  they  were  supposed  to 
consider.  "Medicare  health  plans"  will  be  replaced  with  "Medicare  health  insurance  options," 
and  this  question  will  be  retested  in  Round  2. 

Q31.     Answer  choice  C  (None  of  the  above)  will  be  changed  to  "Neither." 

Q32.     No  problems  were  noted. 

Q33.     "No  matter  which  health  plan  you  choose. . ."  will  be  reworded  to  read  "No  matter  which 

Medicare  health  insurance  plan  you  choose. . .,"  and  this  question  will  be  retested  in  Round  2. 

Q34.     This  question  is  similar  to  Q33  but  includes  a  definition  of  quality  at  the  beginning  of  the 
question.  A  clear  preference  for  whether  or  not  to  include  a  definition  did  not  emerge  from 
testing.  For  the  most  part,  including  the  definition  did  not  affect  respondents'  answers;  however, 
two  respondents  answered  this  question  correctly  but  answered  Q33  incorrectly.  TAP  members 
were  consulted  for  further  guidance  on  this  matter.  The  phrase  health  plan  will  be  replaced  with 
"Medicare  health  insurance  plan"  for  consistency. 

Q35.  Medicare  health  plan  will  be  replaced  with  "Medicare  health  insurance  plan."  This  phrase  was 
tested  in  Round  2. 

Q36.  This  question  was  changed  from  a  Yes/No  to  a  True/False  question.  Question  36b  (The  right  to 
change  plans  as  often  as  you  like)  will  be  dropped.  As  one  respondent  noted,  "it  does  not  make 
sense  that  someone  would  be  able  to  change  plans  every  week  or  even  every  month." 

Q37.  This  question  was  changed  from  a  True/False  to  a  Yes/No  question  to  avoid  asking  the  question 
with  a  "no"  and  having  "false"  as  the  answer. 

Q38.    No  problems  were  noted. 

Q39.  "Hotline"  will  be  taken  out  of  the  question.  It  was  reworded  to  read  "When  you  call  1-800- 
MEDICARE. ..."  One  respondent  indicated  that  she  felt  the  term  "hotline"  was  misleading 
because  it  implies  that  there  is  an  emergency. 

Q40.     Some  respondents  were  confused  by  the  phrase  "information  or  counseling  service."  Others 
thought  that  the  use  of  this  phrase  made  the  correct  answer  obvious.  As  one  respondent  said, 
"Why  would  you  be  asking  me  about  this  if  there  wasn't  an  information  or  counseling  service?" 
This  phrase  was  replaced  with  "service,"  and  this  question  will  be  retested  in  Round  2. 

Q41.  This  question  was  dropped  and  replaced  with  a  new  question  about  the  concept  of  the  Medicare 
&  You  handbook.  The  TAP  committee  was  consulted  to  help  focus  the  question  better. 

Q42.     This  question  was  changed  from  a  True/False  to  a  Yes/No  format. 

Q43.  Several  respondents  asked  the  interviewer  to  repeat  this  question.  A  simplified  version  was 
developed  and  tested  in  Round  2. 

Q44.     No  problems  were  noted. 
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Q45.     This  question  was  dropped  because  we  have  multiple  questions  that  address  this  concept,  and 
other  items  appear  to  be  better  candidates. 

Q46.     Answer  choice  C  (After  a  30-day  waiting  period)  was  dropped. 

Q47.     A  simplified  version  of  this  question  was  developed  and  tested  in  Round  2. 

Q48.     No  problems  were  noted. 

Based  upon  the  results  of  the  first  round  of  testing  and  upon  feedback  from  the  TAP  and  CMS, 
we  revised  the  survey  questions  for  a  second  round  of  testing.  In  addition  to  the  specific  changes  outlined 
above,  we  attempted  to  simplify  the  language  used  in  all  questions  and  to  use  the  same  terminology 
wherever  possible.  We  also  organized  the  questions  into  content  categories  and  added  introductory 
statements  for  each  group  of  questions  to  improve  comprehension.  The  next  chapter  details  the  results  of 
Round  2  of  cognitive  testing. 
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3.1  INTRODUCTION 

In  this  chapter  we  briefly  summarize  key  findings  from  Round  2  of  cognitive  testing  interviews, 
which  were  conducted  between  August  23  and  August  28,  2001.  During  this  round,  we  tested  a  total  of 
106  questions  organized  into  the  following  ten  categories: 

1.  Self -Reported  Knowledge  (n  =  2)  (questions  1  and  102);1 

2.  Eligibility  for  and  Structure  of  Original  Medicare  (n  =  15)  (questions  2  through  12e); 

3.  Medicare  +  Choice  (n  =  16)  (questions  13  through  28); 

4.  Plan  Choice  and  Health  Plan  Decision-Making  (n  =  8)  (questions  29  through  36), 

5.  Information  and  Assistance,  Beneficiary  Rights,  and  Quality  of  Care  (n  =  12)  (questions  37 
through  48); 

6.  Medigap/Employer-Sponsored  Supplemental  Insurance  (n  =  4)  (questions  49  through  52); 

7.  Health  Literacy  Terminology  (n  =  11)  (questions  53  through  63); 

8.  Health  Literacy  Scenarios  (n  =  17)  (questions  64  through  80); 

9.  Cognitive  Abilities  (n  =  6)  (questions  81  through  86);  and 

10.  Non-Knowledge  Items  (n  =  15)  (questions  87  through  101) 

As  shown  above,  we  included  in  Round  2  several  other  types  of  questions  for  testing  in  Round  2 
in  addition  to  the  knowledge  items  that  we  tested  in  Round  1.  First,  we  included  self-reported  knowledge 
items  (necessary  for  evaluation  of  the  psychometric  properties  of  the  demonstrated  knowledge  items). 
Second,  we  developed  health  literacy  items.  We  developed  two  types  of  questions:  1)  questions  designed 
to  measure  beneficiaries'  knowledge  of  terms  associated  with  Medicare  and  health  insurance  (i.e.,  health 
literacy  terminology  questions)  and  2)  questions  designed  to  evaluate  the  ability  of  beneficiaries  to  use 
information  presented  in  the  Medicare  &  You  handbook  to  answer  related  questions  (i.e.,  health  literacy 
scenario  questions).  The  health  literacy  scenarios  that  we  developed  are  from  the  Medicare  &  You 
handbook  (specifically  pages  6-7  [Set  1],  38-39  [Set  2],  63  [Set  3],  59  [Set  4],  and  29  [Set  5]  of  the 
August  2000  version  of  the  handbook).  These  pages  were  given  to  respondents  to  look  at  as  they 
answered  questions  58  through  73. 

Round  2  also  included  six  questions  that  focus  on  beneficiaries'  reading  (n  =  3)  and  memory 
(n  =  3)  skills  as  a  global  measure  of  their  cognitive  abilities.  Previous  research  has  demonstrated  that  the 
three  reading  items  have  a  statistically  significant  effect  on  beneficiary  knowledge  in  a  multivariate 

1    All  question  numbers  refer  to  the  questionnaire  used  in  the  second  round  of  cognitive  testing,  which 
appears  in  Appendix  C. 
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framework  (McCormack  et  al.,  2001).  Therefore,  we  recommended  their  inclusion  for  analytic  purposes. 
Finally,  we  included  sixteen  additional  non-knowledge  questions  that  will  generate  data  on  beneficiaries' 
information-seeking  attitudes  and  behavior,  use  of  Medicare-related  information  and  decision-support 
tools,  and  decision-making  habits.  These  items  may  reveal  reasons  for  differences  in  beneficiaries' 
knowledge  scores,  particularly  in  regard  to  questions  about  their  exposure  to  informational  resources, 
including  the  Medicare  &  You  handbook.  This  type  of  information  could  illuminate  how  existing 
informational  resources  can  be  modified  to  increase  beneficiary  knowledge. 

3.2    RESPONDENT  CHARACTERISTICS 

We  interviewed  eleven  respondents  during  Round  2.  To  meet  OMB  requirements,  four 
respondents  completed  one-half  of  an  interview.  Two  respondents  completed  the  health  literacy  and 
cognitive  sections  only,  while  two  others  completed  the  knowledge  section  only. 

Participants  were  recruited  from  Raleigh-,  Durham-,  and  Chapel  Hill-area  senior  centers,  senior 
housing  facilities,  and  by  word  of  mouth.  In  addition,  flyers  advertising  the  study  were  posted  in  various 
locations  throughout  the  Research  Triangle  area.  To  qualify  for  the  study,  respondents  had  to  have 
Medicare  coverage  and  could  not  have  participated  in  a  health-related  study  during  the  past  six  months. 
Participants  were  chosen  to  reflect  the  diversity  of  the  Medicare  population  with  respect  to  education, 
health  insurance,  race,  age,  and  sex.  Tables  4  and  5  provide  information  about  the  key  characteristics  of 
participants. 

Table  4:  Sociodemographic  and  Health  Insurance  Characteristics  of  Cognitive  Testing 
Participants 


Medicare 

Respondent 

Sex 

Age 

Medigap 

HMO 

Medicaid 

Education 

Ethnicity 

1 

M 

66 

N 

Y(E) 

N 

Graduated  HS 

White 

2 

F 

71 

Y 

N 

N 

Graduated  HS 

Black 

3 

F 

69 

N 

N 

N 

Some  HS 

Black 

4 

F 

77 

N 

N 

N 

Graduated  HS 

White 

5 

M 

70 

N 

Y(E) 

N 

Some  HS 

Black 

6 

M 

72 

Y(E) 

N 

N 

Graduated 
college 

White 

7 

F 

65 

Y(E) 

N 

N 

Some  college 

White 

8 
9 

F 
M 

74 
74 

Y 
Y 

N 
N 

N 
N 

Graduated 
college 

Graduated 
college 

White 
White 

10 

M 

79 

Y 

N 

N 

Graduated  HS 

White 

11 

F 

77 

Y 

N 

N 

Some  college 

White 

Y  =  Yes,  N  =  No,  E  =  Employer-sponsored 
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Table  5:  Summary  of  Sociodemographic  and  Health  Insurance 
Characteristics  of  Cognitive  Testing  Participants  (n  =  11) 


Characteristic 

Number  of  Respondents 

Age 

DO  Da 

o 

70-74 

5 

75-79 

3 

Education 

I        c  than  hinki  cph /~\ I 

Lcoa  iriaii  fiiyn  bonuui 

n 

U 

Some  high  school 

2 

Graduated  high  school 

4 

Some  college 

2 

College  graduate  or  above 

3 

Ethnicity 

Rlark 

O 

White 

8 

Other 

0 

insurance 

Medicare  only 

2 

Medigap 

5 

Employer-sponsored  supplement 

2 

Medicare  HMO 

2 

Medicaid 

0 

3.3  RESULTS 

In  Round  2,  we  tested  a  total  of  53  knowledge  items  and  28  health  literacy  items.  In  Table  6,  we 
report  several  scores  for  each  respondent  by  education  level: 

•  The  number  of  demonstrated  knowledge  items  answered  correctly  (out  of  53), 

•  The  number  of  health  literacy  items  answered  correctly  (out  of  28), 

•  The  number  of  demonstrated  knowledge  and  literacy  items  answered  correctly  (out  of  81), 
and 

•  The  number  of  "Don't  know"  responses  given  by  the  respondent. 

Previous  research  suggests  a  relationship  between  education  level  and  knowledge  of  the  Medicare 
program,  whereby  more  highly  educated  beneficiaries  have  higher  levels  of  knowledge  than  those  with 
less  education  (Hibbard  et  al.,  1998;  Bann  et  al.,  2000).  To  determine  whether  this  relationship  holds  for 
the  new  knowledge  questions,  we  scored  the  demonstrated  knowledge  questions  and  presented  them  by 
education  level.  As  shown  in  Table  6,  the  highest  demonstrated  knowledge  scores  were  received  by  the 
respondents  who  graduated  from  college,  and  the  lowest  demonstrated  knowledge  score  was  received  by 
a  respondent  who  had  some  high  school  education.  However,  there  was  not  a  clear  pattern  among  the 
other  scores. 
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Table  6:  Cognitive  Testing  Participant  Interviewee  Scores  by  Education  Level 


Education 

l^nAiiflarlnA 

i\nowiecige 
Score 
(53  questions) 

neaiin  Literacy 

Score 
(28  questions) 

i  oiai  i\nowieuge 

Score 
(81  questions) 

IMUlTIDer  OT  L/IQ 

Not  Know 
Responses 

Some  High  School 

23/53 
43% 

18/28 
68% 

41/81 
51% 

0 

Some  High  School 

34/53 
64% 

23/28 
82% 

57/81 
70% 

5 

Graduated  High  School 

35/53 
66% 

24/28 
86% 

59/81 
73% 

6 

Graduated  High  School 

29/53 
55% 

21/28 
75% 

50/81 
62% 

3 

Graduated  High  School 

34/53 
64% 

25/28 
89% 

59/81 
74% 

4 

Graduated  High  School 

29/53 
55% 

22/28 
79% 

51/81 
63% 

8 

Some  College 

30/53 
57% 

22/28 
79% 

52/81 
64% 

15 

Some  College 

Skipped 
(1/2  Int.) 

27/28 
96% 

N/A 

2 

Graduated  College 

41/53 
77% 

Skipped 
(1/2  Int.) 

N/A 

9 

Graduated  College 

37/53 
70% 

Skipped 
(1/2  Int.) 

N/A 

5 

Graduated  College 

Skipped 
(1/2  Int.) 

26/28 
93% 

N/A 

0 

Mean  Score 

30.57*  22.14* 
58%  79% 
(9  respondents)  (9  respondents) 

52.71* 
65% 
(7  respondents) 

*  Mean  score  based  on  respondents  who  completed  each  part  of  the  survey. 


Because  many  of  the  new  demonstrated  knowledge  items  contain  only  two  response  options 
(True/False  or  Yes/No),  there  is  a  higher  probability  of  respondents  answering  the  questions  correctly  just 
by  chance.  For  each  of  the  new  knowledge  questions,  however,  respondents  were  given  the  option  of  a 
"Don't  know"  response.  The  concern  about  guessing  is  reduced  if  respondents  in  fact  use  the  "Don't 
know"  response  when  they  feel  they  do  not  know  the  answer.  To  determine  whether  this  was  the  case,  we 
counted  the  number  of  "Don't  know"  responses  for  each  participant  (see  Table  6).  Overall,  it  appears  that 
most  respondents  used  the  "Don't  know"  option  when  they  were  unsure  of  the  answer,  and  some  used  it 
somewhat  liberally.  In  fact,  one  respondent  answered  "Don't  know"  fifteen  times.  There  were,  however, 
two  respondents  who  did  not  choose  "Don't  know"  for  any  of  the  questions. 
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For  the  knowledge  items,  respondents  correctly  answered  between  23  and  41  of  the  53  items,  or 
43  to  77  percent,  with  a  mean  score  of  58  percent.  For  the  health  literacy  items,  respondents  correctly 
answered  between  18  and  27  of  the  28  items,  or  68  to  96  percent,  with  a  mean  score  of  79  percent. 

3.3.1  General  Issues/Findings  by  Question 

The  Round  2  cognitive  testing  protocol  (see  Appendix  C)  was  divided  into  several  sections  that 
coincide  with  the  different  categories  of  questions.  Because  this  was  the  second  round  of  testing,  we 
expected  to  find  fewer,  less  serious  problems  with  the  questionnaire.  General  findings  are  summarized 
below. 

A  handful  of  general  issues  emerged  as  a  result  of  the  second  round  of  testing.  We  also  revisited 
some  issues  studied  in  Round  1.  In  general,  fewer  changes  were  suggested  by  the  second  round  testing 
than  by  the  first,  suggesting  that  the  instrument  worked  better  overall.  Respondents  seemed  to  understand 
most  items  and  their  underlying  concepts  and  had  little  trouble  with  the  questions  and  response  options. 

1.   Our  Round  1  findings  revealed  that,  consistent  with  previous  research,  beneficiaries  had 
difficulty  understanding  the  term  the  Original  Medicare  plan.  Therefore,  in  Round  2  we 
consulted  the  research  of  Fyock  et  al.  (2000)  to  explore  terms  that  could  be  used  as 
substitutes.  Throughout  the  instrument,  we  substituted  the  term  Medicare  Basic  Health 
Insurance  (one  of  the  terms  tested  by  Fyock  et  al.  [2000])  for  the  Original  Medicare  plan. 
This  new  term  did  not  appear  to  create  any  more  or  less  confusion  than  that  found  in  our 
Round  1  experience.  At  the  conclusion  of  the  interviews,  interviewers  read  respondents  the 
definition  of  the  Original  Medicare  plan  from  the  Medicare  &  You  handbook  and  gave  the 
respondents  a  printed  list  of  terms  that  could  be  used  instead: 

Medicare, 

Medicare  Basic  Health  Coverage, 

Medicare  Basic  Medical  Care, 

Traditional  Medicare, 

Regular  Medicare, 

The  Medicare  Basic  Health  Plan,  or 

The  Medicare  Basic  Insurance  Plan. 

No  one  term  was  preferred  by  all  respondents.  As  a  result,  we  opted  to  use  the  phrase  the 
Original  Medicare  plan  to  be  consistent  with  the  Medicare  &  You  handbook.  However,  to 
eliminate  potential  confusion  we  recommend  making  an  electronic  help  screen  available  to 
MCBS  field  interviewers  anytime  that  this  or  other  problematic  terms  appear  in  the 
instrument.  The  help  screen  should  contain  a  useful  definition  of  the  term,  so  that  the 
explanation  given  to  survey  participants  is  consistent  across  interviewers.  The  handbook 
could  be  a  source  for  the  definitions.  We  also  recommend  that  additional  research  be 
conducted  in  this  area  using  larger  sample  sizes. 
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2.  We  also  tested  different  terms  that  can  be  used  to  describe  Medicare  health  plans.  These 
included 

Health  insurance  plans, 
Health  plan  choices, 
Health  plans, 

Medicare  insurance  options, 
Health  insurance  options,  and 
Health  plan  options. 

Again,  no  one  term  was  preferred  by  all  respondents,  however,  there  was  Some  indication  that 
certain  terms  ranked  better  than  the  others.  Three  respondents  preferred  the  term  health  plan 
choices  because  "it's  broader"  and  "implies  options."  Another  respondent  liked  it  "because 
it's  generic  and  patient-friendly."  Three  other  respondents  liked  health  insurance  options  for 
similar  reasons:  "the  word  option  makes  me  think  of  'consumer  power'  and  'freedom  of 
choice.'"  Other  respondents  had  no  strong  preference  or  "no  real  reason"  for  their  choices 
"other  than  how  it  sounds."  We  will  continue  using  the  term  Medicare  health  insurance 
options  until  additional  research  has  been  performed.  We  also  recommend  using  a  help 
screen  for  this  term. 

3.  Testing  again  revealed  that  some  respondents  were  unable  to  distinguish  between  services 
that  are  covered  by  Medicare  and  those  covered  by  other  types  of  insurance  (e.g., 
supplemental  insurance,  Medicaid).  When  asked  whether  or  not  certain  services  were  usually 
covered  by  the  Original  Medicare  plan,  respondents  answered  based  on  whether  the  service 
was  covered  for  them  individually,  considering  their  total  insurance  package.  For  example, 
one  respondent  said  that  "When  I  go  to  the  doctor  I  take  what  they  give  me,  fill  out  anything 
that's  needed,  that's  it.  I  don't  think  about  it  again  unless  I  get  a  bill."  Another  respondent 
had  similar  thoughts  but  added  "A  lot  of  that  insurance  documentation  is  Greek  to  me.  I'd 
have  to  sit  down  and  compare  documents  from  both  insurance  companies  to  figure  it  out,  but 
I  haven't  had  a  reason  to  yet." 

4.  In  general,  using  questions  that  included  "Both"  and  "Neither"  as  response  options  did  not 
present  a  problem  in  terms  of  too  many  response  categories.  However,  in  some  instances 
testing  indicated  that  questions  with  four  response  options  worked  better  if  two  separate 
questions  were  asked  rather  than  one.  For  example,  question  15  ("Imagine  that  a  person  is  in 
a  life-threatening  situation  and  goes  to  the  hospital  emergency  room.  Which  type  of 
Medicare  health  insurance  plan  will  cover  this  visit?  The  Medicare  Basic  Insurance  Plan,  A 
Medicare  HMO,  Both,  Neither")  worked  better  when  broken  into  two  separate  True/False 
questions  about  the  Medicare  Basic  Insurance  Plan  and  a  Medicare  HMO.  This  issue  is  now 
addressed  with  two  questions  instead  of  one. 

5.  In  Round  1,  respondents  were  asked  which  phrase  they  preferred,  "people  on  Medicare"  or 
"people  with  Medicare."  No  clear  preference  emerged.  We  chose  to  use  "people  with 
Medicare"  to  be  consistent  with  the  latest  version  of  the  Medicare  &  You  handbook.  This 
issue  was  addressed  again  in  Round  2  with  the  same  results.  We  reviewed  the  questionnaire 
and  replaced  "people  on  Medicare"  with  "people  with  Medicare"  throughout  the 
questionnaire  to  be  consistent  with  the  Medicare  &  You  handbook. 
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6.  There  was  some  concern  that  the  term  usually  may  be  misread  or  misinterpreted  by  some 
respondents  (e.g.,  one  respondent  read  the  word  as  "unusually").  Substituting  "most  of  the 
time"  lowers  the  reading  level  and  may  help  eliminate  possible  confusion.  We  substituted 
"most  of  the  time"  on  a  case-by-case  basis  when  it  was  an  adequate  replacement. 

7.  We  were  concerned  that  there  may  be  too  many  questions  for  which  the  correct  answer  is 
"True"  (when  unsure,  respondents  are  more  likely  to  answer  'True").  As  a  result,  we  rewrote 
a  small  number  of  questions  to  achieve  a  better  balance  of  True/False  answers,  revising  some 
of  the  questions  to  have  a  Yes/No  format.  However,  it  is  more  important  to  strike  a  balance 
between  the  number  of  questions  with  'True"  or  "False"  answers  and  the  number  with  "Yes" 
or  "No"  answers  administered  in  any  given  year  than  it  is  to  ensure  that  the  entire  item  pool  is 
balanced.  Therefore,  we  strongly  recommend  that  the  balance  of  True/False  and  Yes/No 
questions  be  considered  when  selecting  questions  to  be  administered  each  year. 

3.3.2  Specific  Issues/Findings  by  Question 

In  the  following  section,  we  present  specific  findings  as  they  relate  to  individual  survey  items. 
Where  relevant,  we  include  the  modifications  we  recommended  as  a  result  of  testing.  Please  note  that 
question  numbers  refer  to  the  survey  items  as  numbered  in  the  Round  2  cognitive  testing  protocol, 
provided  in  Appendix  C. 

Ql .      No  problems  were  noted. 

Q2.      No  problems  were  noted. 

Q3.      Most  respondents  knew  that  Medicare  has  two  parts,  but  two  were  unsure  of  what  the  parts  are  or 
which  part  is  hospital  and  which  medical.  One  of  the  respondents  knew  that  Medicare  has  a  part 
that  covers  hospital  costs  and  a  part  that  covers  medical  visits,  but  thought  that  it  might  be  "a  trick 
question"  in  which  they  were  reversed. 

Q4.      Almost  all  respondents  knew  what  a  premium  is  and  answered  the  question  correctly.  One 
respondent  said  that  she  made  a  monthly  payment  but  did  not  know  what  it  is  called. 

Q5.      No  problems  were  noted. 

Q6.      This  question  will  be  dropped.  Nearly  all  respondents  answered  this  question  correctly, 
suggesting  it  was  too  easy  and  therefore  did  not  effectively  test  knowledge. 

Q7.      None  of  the  respondents  had  trouble  understanding  the  intent  of  this  item,  but  several  answered  it 
incorrectly.  Two  respondents  thought  that  this  question  was  referring  to  the  Medicaid  program, 
and  several  others  were  unsure  whether  such  a  program  exists.  However,  the  difficulties  that 
respondents  had  with  this  question  did  not  appear  to  be  due  to  the  comprehensibility  of  the  item 
itself.  Rather,  the  split  between  respondents  answering  correctly  and  those  answering  incorrectly 
suggests  that  this  item  is  a  good  test  of  knowledge. 

Q8.      Most  respondents  answered  this  question  correctly,  suggesting  that  it  is  potentially  too  easy. 
However,  keeping  this  question  item  will  allow  CMS  to  report  that  "[X  %]  of  Medicare 
beneficiaries  do  not  know  that  Medicare  does  not  pay  for  all  health  care  costs,"  which  is 
important  information. 
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Q9.      This  question  was  reworded  to  read  "Medicare  covers  some  preventive  and  screening  services 
such  as  flu  shots  and  tests  for  breast  cancer.  Would  you  say  this  is. . .?" 

Q10  and  Ql  1.  Both  questions  ask  about  "the  Original  Medicare  plan."  We  decided  to  drop  Q10  because 
several  respondents  found  it  "too  long"  or  "confusing."  Question  11,  which  addresses  the  same 
concept,  will  be  kept. 

Q12.     Again,  some  respondents  were  unable  to  distinguish  services  that  are  covered  by  Medicare  from 
those  covered  by  other  types  of  insurance.  When  respondents  were  asked  whether  each  service 
was  usually  covered  by  the  Original  Medicare  plan,  several  respondents  answered  based  on 
whether  the  service  was  covered  for  them  individually,  considering  their  total  insurance  package. 
As  for  the  specific  services  asked  about,  several  respondents  did  not  know  what  "long-term  care 
in  a  nursing  home"  included.  One  respondent  thought  that  long-term  care  was  "about  30  days," 
while  several  others  thought  that  it  was  "indefinite"  or  "as  long  as  necessary."  Therefore,  we 
decided  to  clarify  by  replacing  "long-term  care  in  a  nursing  home"  with  the  more  specific  "a  six- 
month  stay  in  a  nursing  home."  Also,  the  term  Medicare  Basic  Insurance  Plan  was  replaced  with 
the  Original  Medicare  plan.  Testing  indicated  that  the  term  Medicare  Basic  Insurance  Plan  did 
not  work  better  or  worse  than  the  Original  Medicare  plan  (see  Chapter  3.3.1),  so  we  decided  to 
use  the  term  Original  Medicare  plan  throughout  the  questionnaire  to  be  consistent  with  the 
Medicare  &  You  handbook.  We  also  recommend  that  a  help  screen  be  introduced  whenever  the 
term  Original  Medicare  plan  is  used  (see  Chapter  3.3.1). 

Q13,  Q14,  Q15.  Respondents  were  asked  questions  10  and  1 1  about  emergency  services  under  the 

Original  Medicare  plan  (Q13)  and  under  a  Medicare  HMO  (Q14).  Question  15  combined  both 
concepts  into  one  question.  Respondents  were  asked  which  they  preferred,  Q13  and  Q14  or  Q15. 
Although  results  were  mixed,  the  majority  of  respondents  found  two  separate  questions  to  be 
"less  wordy"  and  "easier  to  understand."  Therefore,  Q15  will  be  dropped,  and  we  recommend 
that  Q13  and  Q14  be  asked  in  alternate  years. 

Q16.    This  question  was  reworded  to  read  "Which  health  insurance  option  usually  covers  routine  health 
care  services  that  people  with  Medicare  get  while  traveling  outside  the  United  States?" 

Q17.    This  item  addresses  the  concept  of  Medicare  managed  care.  Approximately  one-half  of  the 

respondents  answered  this  question  incorrectly.  Although  some  respondents  had  heard  the  term 
Medicare  managed  care,  several  were  unsure  of  its  meaning.  One  respondent  thought  that  it 
referred  to  "any  plan  managed  by  Medicare."  Another  respondent  thought  that  it  was  "health  care 
managed  by  a  case  worker."  In  order  to  simplify  this  item,  the  answer  choice  "A  Medigap 
supplemental  insurance  policy"  will  be  replaced  with  "Both/Neither." 

Q18.    This  item  addresses  the  concept  of  Medicare  +  Choice  plans.  Most  of  the  respondents  had  not 

heard  the  term  Medicare  +  Choice  or  were  unsure  what  it  meant.  One  respondent  thought  that  it 
referred  to  "being  able  to  choose  your  doctor."  Another  respondent  thought  that  it  was  "a 
campaign  by  Medicare  to  explain  their  benefits."  Several  respondents  asked  the  interviewer  to 
repeat  the  question  two  or  more  times.  In  order  to  simplify  this  question,  it  will  be  reworded  to 
read:  "People  can  sign  up  for  Medicare  +  Choice  plans  instead  of  the  Original  Medicare  plan. 
(True/False)." 

Q19.     Several  respondents  answered  this  item  incorrectly,  most  likely  because  they  have  not  traveled 
outside  the  United  States  while  on  Medicare.  However,  no  problems  were  noted  with  the 
wording  of  this  question. 
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Q20.     This  question  will  be  reworded  to  read:  "With  a  Medicare  HMO,  people  almost  always  can  see  a 
specialist  without  a  referral  and  the  visit  will  be  covered.  Would  you  say  this  is... True/False." 

Q21.     This  question  will  be  dropped  because  we  feel  that  the  concept  of  "a  doctor  leaving  his  job  at  an 
HMO"  may  have  been  unclear. 

Q22.     No  problems  were  noted. 

Q23.     No  problems  were  noted. 

Q24.     The  term  Medicare  Basic  Insurance  Plan  was  replaced  with  the  Original  Medicare  plan. 

Q25.  This  question  will  be  reworded  to  read:  "If  your  Medicare  HMO  leaves  the  Medicare  program 
and  you  do  not  choose  another  one,  you  will  be  assigned  to  another  HMO.  Would  you  say  this 
is... True/False." 

Q26.  Although  several  respondents  said  they  were  unsure  what  a  "Medicare  Private  Fee-for-Service 
plan"  is,  no  major  problems  were  noted  in  their  ability  to  answer  the  question. 

Q27.     No  problems  were  noted. 

Q28.     No  problems  were  noted. 

Q29.     No  problems  were  noted  with  this  question.  However,  small  changes  were  made  to  each  answer 
option  to  simplify  the  language  and  keep  it  consistent  with  CMS  terminology  (e.g.,  "the  Medicare 
Basic  Insurance  Plan"  will  be  replaced  with  "Original  Medicare";  "A  Medicare  HMO  plan"  will 
be  replaced  with  "A  Medicare  HMO";  "Either"  will  be  replaced  with  "Either  one";  and  "Neither" 
will  be  replaced  with  "Neither  one"). 

Q30.     The  main  concern  with  this  item  is  the  use  of  the  term  Medicare  health  insurance  option. 
Although  a  number  of  alternative  phrases  were  tested,  no  one  term  was  preferred  by  all 
respondents  (see  Chapter  3.3.1).  As  a  result,  we  will  continue  to  use  the  term  Medicare  health 
insurance  option(s)  until  additional  research  has  been  performed. 

Q31.     This  question  was  reworded  to  read:  "Which  type  of  Medicare  health  insurance  option  gives 
people  more  freedom  to  choose  the  doctors  and  hospitals  they  want  to  go  to?"  ("You"  was 
replaced  with  "people.") 

Q32.     This  question  was  dropped  because  of  its  similarity  to  Q30. 

Q33.     No  problems  were  noted. 

Q34.     This  question  was  reworded  to  read:  'The  Original  Medicare  plan  covers  preventive  health  care 
services.  Are  these  same  services  covered  if  a  person  chooses  a  different  Medicare  health 
insurance  option?"  This  is  another  question  in  which  the  term  Medicare  Basic  Insurance  Plan 
was  replaced  with  the  Original  Medicare  plan. 

Q35.     This  question  was  dropped  because  the  concept  is  overly  specific  and  is  not  included  in  the 
handbook. 
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Q36.     This  question  was  reworded  to  read:  "If  you  have  a  Medicare  HMO,  most  likely  you  would  also 
buy  a  Medigap  supplemental  insurance  policy.  Would  you  say  this  is.... True/False."  To  tone 
down  the  language,  we  replaced  the  word  must  with  the  words  mos*  likely. 

Q37.     This  question  was  dropped  because  we  believe  that  it  is  not  a  true  measure  of  knowledge  but 
rather  addresses  "information  seeking." 

Q38  and  Q39.  These  two  questions  asked  about  quality  of  care.  Q39  included  a  definition  of  quality, 
while  Q38  did  not.  Respondents  were  asked  which  version  they  preferred.  Because  no  clear 
preference  emerged,  we  have  retained  both  questions  in  the  item  pool  (see  Chapter  3.3.1). 

Q40.     No  problems  were  noted. 

Q41.     This  question  was  reworded  to  be  a  Yes/No  rather  than  a  True/False  question.  It  now  reads  "If 
you  have  Medicare,  can  your  health  insurance  plan  or  doctor  share  your  health  information 
without  your  permission?" 

Q42.     This  question  was  dropped  because  it  was  too  easy  (eight  of  nine  respondents  answered 
correctly). 

Q43.     This  question  was  reworded  to  read:  "Is  information  about  the  quality  of  care  people  get  with 

different  Medicare  health  insurance  options  available?"  ("You"  was  replaced  with  "people,"  and 
"to  the  public"  was  dropped.) 

Q44.  No  problems  were  noted. 

Q45.  No  problems  were  noted. 

Q46.  No  problems  were  noted. 

Q47.  No  problems  were  noted. 

Q48.  No  problems  were  noted. 

Q49.  No  problems  were  noted. 

Q50.  Several  respondents  who  did  not  have  employer-sponsored  insurance  answered  this  question 
incorrectly.  However,  no  problems  were  noted  with  the  wording  of  this  question. 

Q51.     No  problems  were  noted. 

Q52.     This  question  was  reworded  to  be  a  Yes/No  rather  than  a  True/False  question.  It  now  reads  "Can 
an  insurance  company  refuse  to  sell  you  the  Medigap  supplemental  policy  of  your  choice  during 
the  six  months  after  you  enroll  in  Medicare  Part  B?" 

Health  Literacy  Vocabulary  Questions 

Q53  through  Q63.  In  general,  these  ten  health  literacy  vocabulary  questions  worked  well.  However,  the 
fact  that  most  respondents  answered  a  large  majority  of  the  questions  in  this  section  correctly 
suggests  that  these  questions  may  be  too  easy.  Because  we  only  tested  these  questions  in  the 
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second  round,  additional  testing  is  recommended  before  the  section  is  finalized.  Therefore,  very 
few  changes  were  made  to  these  questions.  Changes  that  were  made  include 

—  One  of  the  answer  options  for  Q58  (generic  drugs)  was  changed  from  "Contain  the  same 
ingredients  as  brand  name  drugs"  to  "Do  work  as  well  as  brand  name  drugs." 

—  An  answer  option  for  Q60  (provider  network)  was  changed  from  "Are  part  of  a  labor 
union"  to  "specialize  in  treating  people  with  certain  diseases." 

It  is  also  important  to  note  that  two  terms — formulary  and  generic  drugs — are  not  included  in  the 
Medicare  &  You  handbook.  Finally,  we  recommend  using  flash  cards  to  show  respondents  these  terms. 


Health  Literacy  Scenario  Questions 

Set  1.  This  set  of  questions  worked  well.  A  few  changes  were  made  based  on  testing: 

1.  A  few  respondents  looked  at  only  the  first  page  of  the  handout.  Therefore,  the  instructions 
were  reworded  to  read:  "Please  look  over  both  pages  6  and  7  and  answer  the  following 
questions." 

2.  Q64:  This  question  was  reworded  to  read:  "If  people  call  1-800-MEDICARE,  they  can 
listen  to  information  in  which  of  the  following  languages?"  "Neither"  was  added  as  an 
answer  option. 

3.  Q65:  Based  on  staff  input,  "imagine  that"  was  added  to  the  beginning  of  this  question.  It 
now  reads:  "Imagine  that  you  call  1-800-MEDICARE  because  you  lost  your  Medicare  card 
and  need  to  get  a  new  one.  Which  number  should  you  press?" 

4.  Q66:  "Imagine  that"  was  added  to  the  beginning  of  this  question,  so  it  now  reads:  "Imagine 
that  you  call  1-800-MEDICARE  and  want  to  speak  with  a  customer  service  representative. 
Which  number  should  you  press?" 

5.  Q67:  This  question  was  dropped  because  it  was  not  considered  a  major  concept. 

Set  2.  While  most  respondents  answered  these  items  correctly,  a  few  respondents  thought  that  this  set 
was  "a  little  busy"  and  "not  as  clear  as  the  first  set."  No  changes  were  made  to  the  instructions  or 
questions  in  this  section. 

Set  3.  Several  changes  were  made  to  this  section: 

1.  A  paragraph  introducing  the  availability  of  local  Medicare  health  plan  information  was 
inserted  at  the  beginning  of  this  section. 

2.  Q72:  This  question  was  reworded  by  adding  "If  you  call  1-800-MEDICARE"  to  the 
beginning.  It  now  reads:  "If  you  call  1-800-MEDICARE,  what  kind  of  information  can  you 
get  from  a  customer  service  representative?" 

3.  Q73:  The  word  "you"  in  this  question  was  replaced  with  "people."  The  question  now  reads: 
"How  can  people  access  the  'Medicare  Personal  Plan  Finder'  ?"  "Neither"  was  also  added  as 
an  answer  option. 
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Set  4.  No  problems  were  noted  in  this  section.  "Neither"  was  added  as  an  answer  option  for  Q74. 

Set  5.  No  problems  were  noted  in  this  section.  The  instructions  were  reworded  for  greater  clarity.  They 
now  read:  "Please  look  over  page  29  and  answer  the  following  questions  according  to  information  in  the 
table." 

Cognitive  Abilities.  No  problems  were  noted  in  this  section.  Three  additional  questions  were  added  to 
the  beginning  of  this  section. 

Non-Knowledge  Items.  No  problems  were  noted  in  this  section.  Question  92  was  reworded  to  read: 
"How  much  do  you  trust  the  information  you  get  from  the  Medicare  program?"  ('The  Medicare 
program"  was  substituted  for  "Medicare.")  Also,  based  on  input  from  project  staff,  Q97  was  dropped 
because  it  is  a  non-knowledge  item  seldom  used  for  analysis. 
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Based  on  the  findings  from  Rounds  1  and  2  of  cognitive  testing,  we  designed  a  questionnaire  that 
includes  a  total  of  99  questions  (See  Appendix  D).  The  Flesch-Kincaid  reading  analysis  score  for  the 
questionnaire  is  at  the  7.5  grade  level.  The  questions  are  broken  down  into  the  following  ten  categories: 

•  Self-Reported  Knowledge  (n  =  3)  (questions  1  through  3); 

•  Eligibility  for  and  Structure  of  Original  Medicare  (n  =  14)  (questions  4  through  13); 

•  Medicare  +  Choice  (n  =  12)  (questions  14  through  25); 

•  Plan  Choice  and  Health  Plan  Decision-Making  (n  =  7)  (questions  26  through  32); 

•  Information  and  Assistance,  Beneficiary  Rights,  and  Quality  of  Care  (n  =  10)  (questions  33 
through  42); 

•  Medigap/Employer-Sponsored  Supplemental  Insurance  (n  =  4)  (questions  43  through  46); 

•  Health  Literacy  Terminology  (n  =  11)  (questions  47  through  57); 

•  Health  Literacy  Scenarios  (n  =  16)  (questions  58  through  73); 

•  Cognitive  Abilities  (n  =  6)  (questions  74  through  79);  and 

•  Non-Knowledge  Items  (n  =  16)  (questions  41,  42,  and  80  through  93). 

The  self-reported  knowledge  items  are  necessary  for  evaluation  of  the  psychometric  properties 
(namely,  the  validity)  of  the  demonstrated  knowledge  items  (which  include  questions  4  through  40  and  43 
through  46).  Health  literacy  may  play  a  role  in  the  ability  of  beneficiaries  to  comprehend  Medicare- 
related  information;  therefore,  we  were  asked  to  develop  questions  in  this  domain.  In  consultation  with  a 
health  literacy  expert,  we  developed  two  types  of  questions.  One  type  measures  beneficiaries'  knowledge 
of  terms  associated  with  Medicare  and  health  insurance  (health  literacy  terminology  questions),  and  the 
other  evaluates  beneficiaries'  ability  to  use  information  presented  in  the  Medicare  &  You  handbook  to 
answer  related  questions  (health  literacy  scenario  questions).  These  scenarios  reference  pages  6  and  7 
(Set  1),  38  and  39  (Set  2),  63  (Set  3),  59  (Set  4),  and  29  (Set  5)  of  the  August  2000  version  of  the 
handbook.  These  pages  must  be  given  to  the  participant  in  order  for  him  or  her  to  complete  questions  58 
through  73. 

We  included  six  questions  that  focus  on  beneficiaries'  reading  (n  =  3)  and  memory  (n  =  3)  skills 
as  a  global  measure  of  their  cognitive  abilities.  Research  has  demonstrated  that  these  three  reading  items 
have  a  statistically  significant  effect  on  beneficiary  knowledge  in  a  multivariate  framework  (McCormack 
et  al.,  2001).  Therefore,  we  recommend  that  they  be  included  for  analytic  purposes.  Finally,  we  included 
sixteen  additional  non-knowledge  questions  that  will  generate  data  on  beneficiaries'  information-seeking 
attitudes  and  behavior,  use  of  Medicare-related  information  and  decision-support  tools,  and  decision- 
making habits.  Responses  to  these  questions  may  help  to  explain  differences  in  beneficiaries'  knowledge 
scores,  particularly  in  regard  to  their  exposure  to  informational  resources,  including  the  Medicare  &  You 
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handbook.  This  type  of  information  could  illuminate  how  existing  informational  resources  can  be 
modified  to  increase  beneficiary  knowledge. 

We  recommend  that  the  next  step  in  the  development  of  the  MCBS  knowledge  items  be  a  pretest 
in  which  all  of  the  newly  developed  demonstrated  knowledge  items  (questions  4  through  40,  43  through 
46)  are  administered  to  a  large  sample  of  respondents.  The  respondents  selected  for  the  pretest  should  be 
representative  of  the  population  to  whom  the  items  will  eventually  be  administered.  Proceeding  in  this 
way  will  allow  us  to  capitalize  on  the  potential  benefits  of  IRT. 

Purpose  of  the  Pretest 

The  primary  purpose  of  the  pretest  is  to  establish  the  IRT  parameters  (i.e.,  difficulty  and 
discrimination)  for  the  items.  This  process  is  also  called  calibration  of  the  items.  Once  the  items  have 
been  calibrated,  we  can  develop  a  set  of  equivalent  forms  that  will  allow  different  sets  of  respondents  to 
receive  different  knowledge  questions,  while  still  receiving  comparable  scores.  Calibration  of  the  items 
would  also  make  it  possible  to  change  the  items  from  year  to  year  and  potentially  to  intersperse  new  items 
during  future  years. 

Pretesting  is  important  for  several  reasons.  During  the  pretest,  all  of  the  knowledge  items  should 
be  administered  to  the  same  group  of  individuals  to  allow  us  to  evaluate  the  relationship  of  the  items  and 
to  examine  how  the  parameters  of  the  items  compare  to  each  other.  For  example,  by  obtaining 
information  on  all  of  the  items,  we  can  determine  whether  a  particular  item  has  a  higher  difficulty  level 
relative  to  the  other  items. 

Pretest  data  can  serve  to  evaluate  the  psychometric  properties  of  the  items,  allowing  us  to  eliminate  or 
modify  any  poorly  performing  items.  The  pretest  data  will  allow  us  to  select  the  best-performing  items 
for  inclusion  in  the  final  knowledge  index.  Specifically,  we  would  include  items  that  demonstrate  high 
levels  of  discrimination  and  varying  levels  of  difficulty. 

Using  the  pretest  data,  we  can  conduct  a  factor  analysis  to  investigate  the  dimensionality  of  the 
scale.  One  of  the  primary  assumptions  of  IRT  is  that  the  scale  being  analyzed  is  unidimensional. 
Although  in  the  development  process  we  have  created  the  items  using  a  set  of  content  areas,  without  data, 
we  will  not  know  how  these  items  will  actually  cluster  when  analyzed.  The  best  IRT  scenario  is  that  the 
entire  set  of  items  will  form  a  unidimensional  scale.  However,  if  we  found  that  the  items  do  not  form  a 
unidimensional  scale,  one  option  would  be  to  use  the  factor  analysis  results  to  assemble  items  into 
subscales.  Another  option  is  to  create  a  cluster  of  related  items  called  a  testlet. 

Conducting  the  Pretest 

Ideally,  to  produce  the  most  reliable  results,  the  pretest  of  the  items  should  be  conducted  using  as 
large  a  sample  as  possible.  Therefore,  the  preferred  option  is  to  field  all  of  the  demonstrated  knowledge 
items  for  the  entire  MCBS  sample.  At  least  some  health  literacy  items  should  be  included,  as  well  as 
some  non-knowledge  items  (even  if  in  a  later  round  of  the  MCBS).  However,  we  recognize  that  this  may 
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not  be  feasible  and  therefore  present  two  alternative  approaches.  To  calibrate  the  items,  we  need  a 
minimum  of  1,000  responses  to  all  of  the  knowledge  items. 

One  approach  is  to  administer  the  entire  item  pool  to  only  those  respondents  who  are  in  their  final 
year  of  the  MCBS.  Since  these  respondents  will  not  be  continuing  in  the  survey,  there  should  be  no 
detrimental  effects,  psychometrically  speaking,  of  exposing  them  to  the  entire  item  pool.  For  example, 
any  concerns  about  the  respondents  learning  the  answers  to  the  questions  would  be  eliminated  because 
they  will  not  be  administered  the  items  again.  However,  one  disadvantage  of  this  approach  is  that  the 
answers  of  respondents  in  their  last  year  of  the  study  may  differ  from  those  of  beneficiaries  who  have  just 
begun  the  study. 

The  other  approach  is  to  randomly  select  a  certain  percentage  of  respondents  to  receive  all  of  the 
knowledge  items.  This  approach  has  the  advantage  of  providing  a  more  representative  sample  of 
respondents.  Without  at  least  1,000  responses  for  a  substantial  portion  of  the  demonstrated  knowledge 
questions,  we  will  be  unable  to  complete  the  second  phase  of  the  project  as  originally  intended. 

In  conclusion,  conducting  a  pretest  and  calibrating  the  new  knowledge  items  will  allow  CMS  to 
benefit  from  the  many  useful  features  of  IRT.  Specifically,  using  IRT  will  produce  a  more  precise 
measure  of  beneficiary  knowledge,  allowing  CMS  to  more  accurately  determine  areas  in  which 
beneficiaries  lack  knowledge  and  could  use  more  information.  In  addition,  an  IRT-based  knowledge 
scale  will  allow  CMS  to  track  knowledge  over  time,  thereby  allowing  for  the  evaluation  of  the 
effectiveness  of  interventions  and  education  programs. 


RTI 


35 


References 


Bann,  C,  K.S.  Lissy,  S.  Keller,  S.A.  Garfinkel,  and  A.J.  Bonito.  2000.  Analysis  of  Medicare  Beneficiary 
Baseline  Knowledge  Data  from  the  Medicare  Current  Beneficiary  Survey:  Knowledge  Index  Technical 
Note.  Report  prepared  for  the  Health  Care  Financing  Administration  under  Contract  No.  500-95- 
0061/004. 

Bann,  CM.  2001.  Item  Response  Theory  Analyses  of  the  Demonstrated  Knowledge  Items  From  the  1999 
Medicare  Current  Beneficiary  Survey.  Report  prepared  for  the  Centers  for  Medicare  and  Medicaid 
Services  under  Contract  No.  500-00-0024/002. 

DeMaio,  T.,  and  J.  Rothgeb.  1996.  "Cognitive  Interviewing  Techniques:  In  the  Lab  and  in  the  Field." 
In  Answering  Questions:  Methodology  for  Determining  Cognitive  and  Communicative  Processes  in 
Survey  Research,  N.  Schwarz  and  S.  Sudman,  eds.,  p.  177.  San  Francisco:  Jossey-Bass. 

Forsyth,  B.,  and  J.  Lessler.  1991.  "Cognitive  Laboratory  Methods:  A  Taxonomy."  hi  Measurement 
Errors  in  Surveys,  P.  Biemer,  R.  Groves,  L.  Lyberg,  N.  Mathiowetz,  and  S.  Sudman,  eds.,  p.  393.  New 
York:  John  Wiley  &  Sons. 

Hibbard,  J.,  J.  Jewett,  S.  Engelmann,  and  M.  Tusler.  1998.  Can  Medicare  beneficiaries  make  informed 
choices?  Health  Affairs  17(6):  181-193. 

McCormack,  L.,  W.  Anderson,  S.  Daugherty,  K.  Ross,  M.  Kuo,  J.  Lynch,  S.  Scheffler,  and  S.  Garfinkel. 
Oct.  15,  2001.  National  Evaluation  of  the  Medicare  &  You  Handbook  Final  Report.  Report  prepared  for 
the  Centers  for  Medicare  and  Medicaid  Services  under  Contract  No.  500-96-00 10T004 


RTI 


37 


Appendix  A. 
Technical  Advisory  Panel  (TAP)  Members 


TAP  Member 

L.  Sue  Andersen 
Ute  Bayen 

Bonnie  Burns 
Joyce  Dubow 

Judith  Hibbard 
Marisa  Scala 
Shoshanna  Sofaer 
David  Thissen 


Affiliation 

HealthNav,  Inc. 

Department  of  Psychology,  University  of  North  Carolina, 
Chapel  Hill 

California  Health  Associates 

Public  Policy  Institute,  American  Association  of  Retired 
Persons 

University  of  Oregon 

Center  for  Medicare  Education 

School  of  Public  Affairs,  Baruch  College 

Department  of  Psychology,  University  of  North  Carolina, 
Chapel  Hill 


Appendix  B. 

MCBS  Knowledge  Questionnaire  Development  (Rnd1) 
Cognitive  Testing  Protocol 
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7964.002.002 


Instructions  for  the  Interviewer 

1.  The  following  are  general  things  to  note  throughout  the  interview: 

•  Note  any  questions  that  don't  work,  even  if  there  is  no  probe.  Make  a  note  of  the  specific 
problem. 

•  Note  terms  that  the  respondent  has  trouble  with  (i.e.,  premium,  out-of-pocket,  etc.) 

•  Note  any  questions  that  the  respondent  says  don't  apply 

•  Note  how  many  times  respondents  ask  for  you  to  repeat  a  question 

•  Note  question  order  problems 

•  Watch  for  difficulty  a  respondent  might  have  with  changing  question  formats  (i.e.,  T/F, 
multiple  choice) 

•  Note  places  where  the  respondent  appears  to  be  guessing 

•  The  first  few  questions  use  the  term  "Medicare"  before  "Original  Medicare"  is  defined.  Note 
whether  the  respondent  is  confused  after  the  transition  to  "Original  Medicare."  Probe  if 
necessary. 

•  Currently  questions  shift  between  using  "people"  and  "you".  Probe  to  determine  if 
respondents  have  a  preference. 

2.  Remind  the  respondent  as  necessary  that  they  can  answer  "Don't  Know". 

3.  If  you  run  into  a  situation  where  the  respondent  says  that  he/she  doesn't  know  what 
something  is  (for  example,  Medicare  +  Choice)  and  this  prevents  him/her  from  answering 
subsequent  questions,  explain  that  it's  important  to  answer  every  question  and  that  he/she 
might  remember  as  you  go  along.  If  they  have  major  difficulty  in  any  area,  just  make  a  note 
of  it  an  move  on. 

4.  If  the  respondent  has  questions  about  Medicare  refer  them  to  state-level  Senior  Health 
Insurance  Counseling  Program  (SHIP)  at  800-443-9354  or  919-733-01 11.  You  may  need  to 
be  firm  in  this  regard,  otherwise  you  may  lose  precious  time. 

5.  Score  questionnaire  when  complete  to  calculate  a  number  (out  of  55)  and  a  percentage  (out 
of  100)  correctly  answered. 

6.  As  you  conduct  the  interview,  you  should  read  the  entire  question  and  all  responses. 
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Introduction 

To  be  read  to  the  respondent: 

Before  we  get  started,  I  want  to  give  you  a  little  background  information  about  what  we're 
doing.  We  have  developed  a  survey  for  people  on  the  Medicare  program.  The  purpose  of  this 
type  of  interview  is  for  us  to  see  how  these  questions  work.  Do  they  make  sense?  Are  they  being 
understood  as  we  planned?  How  easy  or  difficult  are  they  to  answer?  We  want  to  understand 
what  you  think  each  question  means  and  how  you  arrive  at  your  answer.  This  will  help  us  find 
potential  problems  with  the  questions  so  that  we  can  make  them  better  before  we  survey  a  lot  of 
people.  I  am  not  an  expert  in  health  insurance  counseling  and  am  not  familiar  with  your 
insurance  situation,  so  unfortunately,  I  won 't  be  able  to  help  you  with  any  questions  that  you 
may  have  about  your  health  insurance. 

I  will  be  reading  the  questions  and  answer  choices  out  loud.  After  many  of  the  items  I  will  ask 
you  some  additional  questions  to  find  out  what  you  think  about  the  questions.  We  are  more 
interested  in  your  interpretation  of  the  questions  than  your  actual  answers.  If  you  don't  know 
the  answer  to  any  of  the  questions  you  can  always  say  "don't  know".  You  can  also  refuse  to 
answer  any  questions  and  we  can  stop  at  any  time.  The  interview  should  take  approximately  2 
hours  to  complete.  Do  you  have  any  questions? 

Now  I'm  going  to  give  you  an  "Informed  Consent"  form  for  you  to  read.  This  explains  the  study 
and  what  you  are  being  asked  to  do.  Please  take  a  few  minutes  to  review  and  sign  on  the 
signature  line  indicating  that  you  understand  your  role  and  are  participating  voluntarily.  I  will 
also  give  you  another  copy  of  the  form  for  you  to  keep.  There  are  phone  numbers  of  researchers 
at  RTI  who  you  can  call  if  you  have  any  questions  about  the  study  after  today. 

[Explain  where  restrooms  are  if  necessary.] 
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I. 


Self-Perceived  Knowledge.  This  first  section  asks  about  how  you  feel  about  your 
own  understanding  of  the  Medicare  program. 


1.  How  do  you  rate  your  understanding  of  the  Medicare  program? 


a. 


Excellent 


□  i 

□  2 

□  3 

□  4 

□  5 


b. 


c. 


Very  good 
Good  


d. 


Fair 


e. 


Poor 


II.       Eligibility  for  and  Structure  of  Original  Medicare  (including  covered  benefits,  costs, 
and  billing  issues).  This  section  asks  questions  about  what  Medicare  is,  who  is 
eligible  for  Medicare,  how  Medicare  works,  and  what  types  of  services  are  covered. 

2.  People  are  eligible  for  Medicare  because  they  have  low  or  moderate 
incomes.  Would  you  say  this  is. . . 

a.  True   □  j 

b.  False   Q2 

c.  Don't  know  

Probe:  In  your  own  words,  what  does  "eligible  for  Medicare"  mean? 
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3.  Medicare  has  two  parts.  Part  A  is  hospital  insurance.  Part  B  is  medical 
insurance.  Would  you  say  this  is. . . 

a.  True   □  j 

b.  False   Q2 

c.  Don't  know....  


4.  Most  people  on  Medicare  have  to  pay  a  monthly  payment,  called  a  premium. 
Would  you  say  this  is. . . 

a.  True   □  , 

b.  False   □  2 

c.  Don't  know  


5.  The  Medicare  Part  B  monthly  payment,  called  a  premium,  can  change  at  any 
time  during  the  year.  Would  you  say  this  is. . . 

a.  True   □ , 

b.  False   □  2 

c.  Don't  know   Q.j 

[INTERVIEWER:  I)  Does  the  respondent  have  difficulty  with  the  transition 
from  just  "Medicare  "  to  "the  Original  Medicare  plan  "  ?  2)  Does  the 
respondent  have  trouble  with  the  long  response  option  7c?] 
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6.  The  Original  Medicare  plan  is. . . 

a.  Medicare  Part  A  only   □ , 

b.  A  Medicare  option  that  is  no  longer  available   □  2 

c.  The  traditional  Medicare  plan  that  has  been  available  since 

Medicare  started   D  3 

d.  Don't  know  

III.     Define  Original  Medicare  for  respondent:  The  Original  Medicare  Plan  is  offered  by 
the  federal  government  and  is  available  nationwide.  You  are  usually  charged  a  fee 


for  each  health  care  service  or  supply  you  get. 

7.  The  Original  Medicare  plan  is  no  longer  available  to  people  on  Medicare. 
Would  you  say  this  is... 

a.  True 

b.  False 

c.  Don't  know 

Probe:  Please  tell  me  how  you  chose  your  answer.  What  did  you  have  to  think 
about? 
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8.  Which  one  of  the  following  statements  is  true  about  what  the  Original 
Medicare  plan  pays  for?  It  pays  for. . . 

a.  All  health  care  costs   □  { 

b.  Some  health  care  costs   [~]  2 

c.  Only  emergency  health  care  costs   □  3 

d.  Don't  know   Q.j 


9.  Medicare  now  pays  for  certain  cancer  screening  services  such  as 

mammograms  (breast  x-rays  for  women)  and  Pap  smears.  Would  you  say 
this  is... 

a.  True   □  j 

b.  False   rj2 

c.  Don't  know   Q.j 


10.  Is  emergency  health  care  paid  for  under  the  Original  Medicare  plan? 

a.  Yes   □ , 

b.  No   \J2 

c.  Don't  know  
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1 1.  Is  emergency  health  care  paid  for  under  a  Medicare  HMO  plan? 

a.   Yes   Q  j 

b-  No   Q2 

c.  Don't  know   Q.j 

Probe:  Please  think  about  the  last  two  questions  I  asked  you.  Now  I'm  going  to 
read  you  an  alternate  question  that  might  be  used  instead  of  the  two  you  just 
answered. 

[INTERVIEWER:  Read  question  slowly] 
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12.  When  a  person  thinks  they  are  in  a  life-threatening  situation,  he/she  can  go 
to  the  nearest  hospital  emergency  room  and  the  visit  will  be  covered.  Does 
this  best  describe... 

a.  The  Original  Medicare  plan   □  j 

b.  A  Medicare  HMO   Q2 

c.  Both   Q3 

d.  Don't  know   {2-\ 

Probe:  Please  think  "aloud"  as  you  answer  this  question.  How  did  you  choose 
your  answer?  What  did  you  have  to  think  about? 


Please  think  about  all  three  questions.  [INTERVIEWER:  You  may  need  to  re- 
read the  question]  What  is  the  difference  between  questions  10,  11  and  12? 
[INTERVIEWER:  Make  sure  the  respondent  understands  that  you  are  asking 
about  10  and  11  as  a  group  compared  to  question  12.] 

Probe:  Which  do  you  prefer?  Why? 

[INTERVIEWER"  Also  probe  to  determine  whether  respondents  have  trouble 
with  the  answer  choice  "both"  in  question  12.] 
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13.  If  you  get  health  care  services  outside  the  United  States,  they  will  be  paid  for 


by... 

a.  The  Original  Medicare  plan   □ , 

b.  A  Medicare  HMO   [j2 

c.  Both   rj3 

d.  Neither   Q4 

e.  Don't  know   Q  j 


Probe:  Please  think  "aloud"  as  you  answer  this  question.  How  did  you  choose 
your  answer?  What  did  you  have  to  think  about? 

[INTERVIEWER:  Probe  to  determine  whether  respondents  have  difficulty  with 
the  answer  choices  "both"  and  "neither".  Does  also  having  "neither"  make  it 
more  difficult?] 


14. 1  am  going  to  read  you  a  list  of  services.  For  each  item,  please  say  whether 
you  think  it  is  covered  by  the  Original  Medicare  plan. 


Yes 

No 

DK 

a. 

Routine  eye  exams  

  □] 

□  2 

□  l 

b. 

Annual  dental  exams  

  Di 

□  2 

□  -1 

c. 

Prescription  drugs  (outside  the  hospital)  

  Di 

□  2 

□  -1 

d. 

Care  in  a  nursing  home  

  Di 

□  2 

□  -1 
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IV.     Managed  Care  and  the  M+C  Program 
*Before  reading  next  question: 

Before  we  go  to  the  next  question,  please  tell  me  in  your  own  words  what 
"managed  care"  means  to  you?  [INTERVIEWER:  Take  verbatim  notes) 


OK,  now  I'm  going  to  read  the  next  question.  [INTERVIEWER:  If  the 
respondent  did  not  know  the  answer  and  says  he/she  can 't  answer  the  question, 
explain  that  they  may  remember  something  by  thinking  about  the  question  and 
answer  choices.] 

INTERVIEWER:  See  if  they  have  problem  remembering  all  the  responses.  Do 
they  ask  you  to  repeat  them  more  than  once? 


15.  Medicare  managed  care  is.... 

a.  An  HMO-type  health  plan   □  j 

b.  A  fee-for-service  health  plan   Q2 

c.  A  Medigap  or  supplemental  insurance  policy   Q3 

d.  Don't  know   □_, 

*Before  reading  next  question: 

Before  we  go  to  the  next  question,  please  tell  me  in  your  own  words  what 
"Medicare  +  Choice"  means  to  you?  [INTERVIEWER:  Take  verbatim  notes} 


OK,  now  I'm  going  to  read  the  next  question.  [INTERVIEWER:  If  the 
respondent  did  not  know  the  answer  and  says  he/she  can 't  answer  the  question 
explain  that  they  may  remember  something  by  thinking  about  the  question  and 
answer  choices.] 
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16.  "Medicare  +  Choice"  refers  to  Medicare  plans. . . 

a.  That  are  for  people  who  want  to  have  choices  when  they  are 

traveling   Di 

b.  That  people  can  sign  up  for  instead  of  the  Original  Medicare  plan   Q2 

c.  For  people  who  have  specific  health  conditions   Q3 

d.  Don't  know  


17.  Medicare+Choice  refers  to  Medicare  plans  that  people  can  sign  up  for 
instead  of  the  Original  Medicare  plan.  Would  you  say  this  is. . . 

a.  True   q  , 

b.  False   □  2 

c.  Don't  know   Q  j 


18.  In  general,  with  a  Medicare  HMO,  you  can  go  to  any  doctor  or  hospital  in 
the  United  States.  Would  you  say  this  is. . . 

a.  True  

b.  False   q2 

c.  Don't  know   Q  j 
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19.  In  general,  with  a  Medicare  HMO,  you  do  not  need  to  get  a  referral  to  see  a 
specialist.  Would  you  say  this  is. . . 

a.  True   □ } 

b.  False   Q2 

c.  Don't  know   Q.j 

Probe:  What  does  referral  mean  to  you  in  this  context? 


20.  During  which  times  can  a  doctor  leave  a  Medicare  HMO? 

a.  Anytime  during  the  year   □  j 

b.  At  the  beginning  of  the  year   Q2 

c.  Never   Q3 

d.  Don't  know   Q_, 

Probe:  Thinking  about  the  answer  choices  to  this  question,  "Anytime  during  the 
year",  "at  the  beginning  of  the  year",  and  "never",  how  easy  or  difficult  were 
they  to  use?  Can  you  think  of  other  answer  choices  that  would  make  more  sense 
or  be  easier  to  use? 


Probe:  If  you  were  asked  this  question  and  were  not  given  answer  choices,  how 
would  you  respond? 
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21.  Once  you  join  a  Medicare  HMO,  the  plan  can  change  your  monthly 
premium... 

a.  Anytime  during  the  year   □  x 

b.  At  the  beginning  of  the  year   □  2 

c.  Never   Q3 

d.  Don't  know   □_! 


22.  If  your  Medicare  HMO  stops  serving  people  with  Medicare  in  your  area,  can 
you  join  another  Medicare  HMO  in  your  area? 

a.  Yes   □  j 

b.  No   Ui 

c.  Don't  know  

Probe:  In  your  own  words,  what  do  you  think  this  question  is  asking? 

Probe:  What  did  you  have  to  think  about  in  order  to  answer  this  question  ?  Are 
you  thinking  "in  general "  or  about  your  specific  situation  ?  AND/OR  What  did 
you  think  about  when  I  said  "in  your  area  "  ? 


23.  If  your  HMO  leaves  the  Medicare  program  and  you  do  not  choose  another 
one,  you  will  be  covered  by  the  Original  Medicare  plan.  Would  you  say  this 
is... 

a.  True   □  j 

b.  False   Q2 

c.  Don't  know   Q.j 
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V.       Medigap-Specific  Issues.  This  section  asks  questions  about  supplemental  insurance, 
sometimes  called  "Medi-gap"  plans. 


24.  People  with  Medicare  coverage  can  always  buy  a  Medigap  or  supplemental 
health  insurance  policy  regardless  of  their  health.  Would  you  say  this  is. . . 

a.  True   □  j 

b.  False   Q2 

c.  Don't  know   rj  j 

Probe:  Please  tell  me  which  phrase  you  prefer:  "People  with  Medicare  "  or 
"People  on  Medicare"  [Re-read  the  question  if  necessary].  Why? 


Probe:  What  did  you  think  about  when  I  said  "Medigap  or  supplemental  health 
insurance  policy  "  ?  Were  those  terms  confusing  ? 
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VI.      Similarities  and  Differences  Between  Plan  Choices/Health  Plan  Decision  Making. 
This  section  asks  questions  about  the  different  health  plan  choices  people  on 
Medicare  have. 


Probe:  What  do  "regular  Medicare  benefits"  mean  to  you  in  this  context? 

Probe:  Have  you  heard  the  term  "Private  Fee-for-Service  plan  "  ?  What  do  you 
think  it  is? 


Probe:  How  did  you  choose  your  answer?  What  did  you  have  to  think  about? 
[INTERVIEWER:  Note  how  "all  of  the  above"  works.  How  many  times  did  the 
respondent  ask  you  to  repeat  the  answers? 


26.  No  matter  what  Medicare  health  plan  you  choose,  your  out-of-pocket  costs 
will  be  the  same.  Would  you  say  this  is. . . 

a.  True   □  j 

b.  False   □  2 

c.  Don't  know  

Probe:  What  does  "out-of-pocket  costs"  mean  to  you  in  this  context? 


25.  People  can  receive  their  regular  Medicare  benefits  through. . . 

a.  The  Original  Medicare  plan  

b.  A  Medicare  HMO  plan  

c.  A  Private  Fee-for-Service  plan  

d.  All  of  the  above  

e.  Don't  know  


□  i 

□  2 

□  3 

□  4 

□  -1 
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27.  With  Medicare  today,  you  have  to  change  health  plans  every  year.  Would 
you  say  this  is... 

a.  True   □  j 

b.  False   Q2 

c.  Don't  know   Q  j 

Probe:  If  this  question  was  re-worded  to  read:  "With  Medicare  today,  people 
have  to  change  health  plans  every  year"  would  its  meaning  change  to  you?  [If 
yes,  how]? 


Probe:  Which  version  do  you  prefer?  Why? 

[INTERVIEWER:  There  are  several  questions  that  use  either  "you"  or 
"people".  Probe  to  find  out  which  the  respondent  prefers.  If  necessary,  probe 
additional  applicable  questions.] 


RTI 


56 


RTI  Appendix  B 


28.  Which  type  of  health  insurance  option  gives  you  more  freedom  to  choose  the 
doctors  and  hospitals  you  want  to  go  to? 

a.  The  Original  Medicare  program   Q , 

b.  A  Medicare  HMO   [j2 

c.  They  are  both  about  the  same   Q3 

d.  Don't  know   [j  A 

[INTERVIEWER:  This  is  a  VERY  important  research  question.  Have  a  full 
conversation  with  the  participant  about  it  and  ask  additional  probes  if 
necessary  and  record  verbatim  responses.  We  are  trying  to  get  at  the  best  way 
to  say  that  there  are  different  ways  that  beneficiaries  can  receive  their 
Medicare  benefits  and  that  they  differ  on  certain  features,  like  choice  of 


doctors.  Is  it  better  to  say  "health  insurance  options"  or  "health  plan  options" 
or  "plan  choices"  or  "Medicare  plans"  or  something  else?  Does  having  the 
word  "insurance"  in  the  phrase  help  them?] 

Probe:  In  your  own  words,  what  do  you  think  this  question  is  asking? 


Probe:  How  did  you  choose  your  answer?  What  did  you  have  to  think  about? 


Probe:  What  does  "health  insurance  option"  mean  to  you  in  this  context? 


Probe:  I'm  going  to  read  some  alternate  phrases  that  could  be  used  to  replace 
"health  insurance  options".  Please  tell  me  what  you  think  of  each  one  and 
which  phrase  you  prefer. 

1.  Health  insurance  plans 

2.  Health  plan  choices 
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3.  Health  plans 

4.  Medicare  health  plans 

5.  Medicare  insurance  options 

Probe:  Are  there  any  other  phrases  that  you  can  think  of  that  would  be  better? 


29.  Which  type  of  health  insurance  option  is  more  likely  to  pay  for  prescription 
drugs? 

a.  The  Original  Medicare  plan   □  x 

b.  A  Medicare  HMO   Q2 

c.  They  are  both  about  the  same   Q3 

d.  Don't  know  


30.  The  Original  Medicare  plan  now  pays  for  more  preventive  health  care 

services  like  diabetes  monitoring.  Are  these  same  services  covered  by  other 
Medicare  health  plans? 

a.  Yes   □  j 

b.  No   \J2 

c.  Don't  know   Q.j 

[INTERVIEWER:  Note  if  you  think  the  person  chose  answer  "c"  because  it  was 
the  last  response  available  or  it  seemed  like  the  right  answer  b/c  it  did  not  have 
a  parallel  negative  choice  ?] 
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31.  Which  of  the  following  people  have  to  change  health  plans  this  year? 

a.  Everyone  who  just  joined  Medicare  last  year   □  j 

b.  Everyone  who  has  a  Medicare  HMO   □  2 

c.  None  of  the  above   Q3 

d.  Don't  know   Q.j 


Probe:  What  kinds  of  "preventive  health  care  services"  are  you  thinking 
about  ?  ASK  ONLY  IF  TIME 


32.  If  you  have  a  Medicare  HMO,  it  is  important  for  you  to  also  buy  a 

supplemental  insurance  policy,  sometimes  called  a  Medigap  policy.  Would 


you  say  this  is... 

a.  True   □ } 

b.  False   Q2 

c.  Don't  know  
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VII.    Differences  in  Quality  of  Care.  This  section  asks  about  quality  of  care. 


33.  No  matter  which  plan  you  choose,  you  get  the  same  quality  of  health  care. 
Would  you  say  this  is. . . 

a.  True   □ ! 

b.  False   Q2 

c.  Don't  know  

[INTERVIEWER:  Now  I'm  going  to  read  you  a  similar  version  of  the  previous 
question.] 

34.  Good  quality  care  is  safe,  effective  and  responsive.  All  health  plans  provide 
the  same  quality  of  care.  Would  you  say  this  is. . . 

a.  True  

b.  False   Q2 

c.  Don't  know   □.j 

Probe:  Is  there  a  difference  between  these  two  questions?  If  yes,  what?  Which 
one  is  easier  for  you  to  understand?  Why? 


Probe:  The  second  version  of  this  question  defines  quality  as  "safe,  effective, 
and  responsive  ".  Did  including  the  definition  affect  your  answer?  How? 

Probe:  What  do  you  think  of  the  definition?  Can  you  think  of  an  alternate  way 
to  define  quality  that  would  make  more  sense  to  you?  ASK  ONLY  IF  YOU 
HAVE  TIME. 
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VIII.    Beneficiaries  Have  Rights  and  Protections.  This  section  acks  questions  about  the 
rights  and  protections  that  people  on  Medicare  have. 

35.  You  have  the  right  to  ask  a  Medicare  health  plan  to  reconsider  any  decision 
it  makes  about  what  you  or  your  doctor  think  should  be  covered.  Is  this  true 


for. . . 

a.  The  Original  Medicare  program   Qj 

b.  Medicare  HMOs   rj2 

c.  Both   rj3 

d.  Neither   rj4 

e.  Don't  know   Q.j 


Probe:  What  does  "reconsider"  mean  to  you  in  this  question? 


[INTERVIEWER:  Note  how  this  question  and  the  answer  choices  "both"  and 
"neither"  work]. 


36.  When  you  are  on  Medicare,  you  have  which  of  the  following  rights?  The 
right  to.... 


Yes 

No 

DK 

a. 

Participate  in  your  treatment  decisions  

  Di 

□  2 

□  l 

b. 

The  right  to  change  plans  as  often  as  you  like  

  Oi 

□  2 

□  1 

c. 

Have  your  health  information  kept  private  

  Di 

□  2 

□  1 
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IX.      How  to  Get  More  Information/Assistance.  This  section  asks  about  ways  to  get  more 
information  or  assistance  about  Medicare. 


37.  Information  about  the  quality  of  care  different  Medicare  plans  give  is  not 
available  at  this  time.  Would  you  say  this  is. . . 

a.  True   □  j 

b.  False   □  2 

c.  Don't  know   rj.j 
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38.  Does  the  Medicare  program  have  its  own  internet  web  site? 

a.  Yes   □  j 

b.  No   Ui 

c.  Don't  know   Q_, 


39.  When  can  you  call  the  1-800-MEDICARE  hotline  to  speak  to  someone  at 
Medicare  about  your  questions? 

a.  Only  during  business  hours   □  l 

b.  Only  on  the  weekends   □  2 

c.  Anytime   Q3 

d.  Don't  know   □_, 


40.  As  far  as  you  know,  is  there  an  information  or  counseling  service  in  your 
area  that  people  with  Medicare  can  use  to  get  help  understanding  and 
comparing  health  insurance  options? 

a.  Yes   □ , 

b.  No   Q2 

c.  Don't  know   □_, 

[INTERVIEWER:  Note  how  this  question  works.  Is  it  too  "wordy"  ?] 
Probe:  [If  yes]  What  kind  of  service  are  you  thinking  about? 
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41.  Does  the  Medicare  program  have  a  handbook  that  tells  you  about  how  the 
Medicare  program  works? 

a.  Yes   □  j 

b.  No   U2 

c.  Don't  know   □_! 

Probe:  [If  yes]  What  is  this  handbook  called?  How  did  you  get  it?  ASK  ONLY 
IF  YOU  HAVE  TIME. 


42.  The  Medicare  program  has  recently  begun  to  offer  more  information  and 
help  in  order  to  answer  questions  about  the  Medicare  program.  Would  you 
say  this  is... 

a.  True   □  j 

b.  False   Q2 

c.  Don't  know  

Probe:  What  kinds  of  "information  and  help  "  are  you  thinking  about? 


X.       Special  Subgroup  Issues. 

[ASK  Q43  ONLY  IF  THE  RESPONDENT  HAS  MEDICAID] 
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43.  States  have  programs  that  pay  some  or  all  of  Medicare's  premiums  for 
people  with  lower  incomes  and  resources.  Would  you  say  this  is. . . 


a. 


True 


□  i 

□  2 

□  -1 


b 


False 


c 


Don't  know 


[ASK  THE  REMAINING  QUESTIONS  ONLY  IF  THE 
RESPONDENT  HAS  HEALTH  INSURANCE  THROUGH  AN 
EMPLOYER] 


44.  People  on  Medicare  who  also  have  health  insurance  from  an  employer  may 
choose  whether  Medicare  or  the  employer  will  be  their  primary  insurer. 
Would  you  say  this  is. . . 

a.  True   □  j 

b.  False   Q2 

c.  Don't  know  


45.  Is  it  a  good  idea  to  take  Medicare  Part  B  coverage  if  you  are  working  after 
age  65  and  have  health  insurance  with  your  employer? 

a.  Yes   □! 


b. 


No 


□  2 

□  -1 


c 


Don't  know 
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46.  If  you  have  a  supplemental  insurance  policy  and  dropped  it,  when  could  you 
get  it  back? 

a.  At  any  time   □  j 

b.  Only  under  certain  conditions   Q  2 

c.  After  a  30  day  waiting  period   □  3 

d.  Never   rj4 

e.  Don't  know  


47.  If  you  enroll  in  Medicare  Part  B  while  you  are  covered  under  an  employer 
plan,  you  may  not  be  able  to  buy  a  Medigap  policy  when  your  employer 
coverage  stops.  Would  you  say  this  is. . . 

a.  True   □  j 

b.  False   Q2 

c.  Don't  know   □ 
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XI.      Self-Perceived  Knowledge.  Finally,  this  last  question  asks  you  how  much  you  feel 
that  you  know  about  the  Medicare  program. 


48.  How  much  do  you  think  you  know  about  the  Medicare  program? 

a.  I  know  just  about  everything  I  need  to  know   □  j 

b.  I  know  most  of  what  I  need  to  know   rj  2 

c.  I  know  a  little  of  what  I  need  to  know   □  3 

d.  I  know  almost  none  of  what  I  need  to  know   Q4 
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Summary  Questions 

1.  Other  than  what  we've  discussed,  were  there  any  questions  or  terms  that  were  confusing  or 
difficult? 


2.  Overall,  how  easy  or  difficult  was  it  for  you  to  complete? 


3.  How  often  did  you  guess  at  an  answer?  Can  you  remember  any  specific  questions  where  you 
guessed? 


4.  How  confident  are  you  that  you  were  able  to  answer  most  questions  correctly? 


5.  Where  do  you  get  information  about  Medicare  (i.e.,  written  materials,  phone  line,  web,  etc.)? 
ONLY  IF  THERE'S  TIME 
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Appendix  C. 

MCBS  Knowledge  Questionnaire  Development  (Rnd2) 
Cognitive  Testing  Protocol 

August  21,  2001 
7964.002.002 

Instructions  for  the  Interviewer 

1.  The  following  are  general  things  to  note  throughout  the  interview: 

•  Note  any  questions  that  don't  work,  even  if  there  is  no  probe.  Make  a  note  of  the  specific 
problem. 

Note  terms  that  the  respondent  has  trouble  with  (i.e.,  premium,  out-of-pocket,  etc.) 
Note  any  questions  that  the  respondent  says  don't  apply 
Note  how  many  times  respondents  ask  for  you  to  repeat  a  question 
Note  question  order  problems 

•  Watch  for  difficulty  a  respondent  might  have  with  changing  question  formats  (i.e.,  T/F, 
multiple  choice) 

•  Note  places  where  the  respondent  appears  to  be  guessing 

2.  Remind  the  respondent  as  necessary  that  they  can  answer  "Don't  Know". 

3.  If  you  run  into  a  situation  where  the  respondent  says  that  he/she  doesn't  know  what 
something  is  (for  example,  Medicare  +  Choice)  and  this  prevents  him/her  from  answering 
subsequent  questions,  explain  that  it's  important  to  answer  every  question  and  that  he/she 
might  remember  as  you  go  along.  If  they  have  major  difficulty  in  any  area,  just  make  a  note 
of  it  an  move  on. 

4.  If  the  respondent  has  questions  about  Medicare  refer  them  to  state-level  Senior  Health 
Insurance  Counseling  Program  (SHIP)  at  800-443-9354  or  919-733-01 11.  You  may  need  to 
be  firm  in  this  regard,  otherwise  you  may  lose  precious  time. 

5.  Score  questionnaire  when  complete  to  calculate  a  number  and  percentage. 

6.  As  you  conduct  the  interview,  you  should  read  the  entire  question  and  all  responses. 
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Introduction 

To  be  read  to  the  respondent: 

Before  we  get  started,  I  want  to  give  you  a  little  background  information  about  what  we  're 
doing.  We  have  developed  a  survey  for  people  on  the  Medicare  program.  The  purpose  of  this 
type  of  interview  is  for  us  to  see  how  these  questions  work.  Do  they  make  sense?  Are  they  being 
understood  as  we  planned?  How  easy  or  difficult  are  they  to  answer?  We  want  to  understand 
what  you  think  each  question  means  and  how  you  arrive  at  your  answer.  This  will  help  us  find 
potential  problems  with  the  questions  so  that  we  can  make  them  better  before  we  survey  a  lot  of 
people.  I  am  not  an  expert  in  health  insurance  counseling  and  am  not  familiar  with  your 
insurance  situation,  so  unfortunately,  I  won 't  be  able  to  help  you  with  any  questions  that  you 
may  have  about  your  health  insurance. 

I  will  be  reading  the  questions  and  answer  choices  out  loud  (Note:  In  some  sections  respondents 
will  be  given  the  questions  and  answers  to  look  at).  After  many  of  the  items  I  will  ask  you  some 
additional  questions  to  find  out  what  you  think  about  the  questions.  We  are  more  interested  in 
your  interpretation  of  the  questions  than  your  actual  answers.  If  you  don't  know  the  answer  to 
any  of  the  questions  you  can  always  say  "I  don 't  know".  You  can  also  refuse  to  answer  any 
questions  and  we  can  stop  at  any  time.  The  interview  should  take  approximately  2  hours  to 
complete.  Do  you  have  any  questions? 

Now  I'm  going  to  give  you  an  "Informed  Consent"  form  for  you  to  read.  This  explains  the  study 
and  what  you  are  being  asked  to  do.  Please  take  a  few  minutes  to  review  and  sign  on  the 
signature  line  indicating  that  you  understand  your  role  and  are  participating  voluntarily.  I  will 
also  give  you  another  copy  of  the  form  for  you  to  keep.  There  are  phone  numbers  of  researchers 
at  RTI  who  you  can  call  if  you  have  any  questions  about  the  study  after  today. 

[Explain  where  restrooms  are  if  necessary.] 
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I.        Self-reported  knowledge  question 

1.  How  easy  or  difficult  is  the  Medicare  program  to  understand?  Would 
you  say  it  is... 

a.  Very  easy  to  understand   □ , 

b.  Somewhat  easy  to  understand   □  2 

c.  Somewhat  difficult  to  understand   Q3 

d.  Very  difficult  to  understand   Q4 

II.      Eligibility  for  and  Structure  of  Original  Medicare 

2.  People  are  eligible  for  Medicare  because  they  have  low  or  moderate 
incomes.  Would  you  say  this  is. . . 

a.  True   [Jx 

b.  False   rj2 

c.  Don't  know  

Probe:  In  your  own  words,  what  does  "eligible  for  Medicare  "  mean? 

Probe:  If  this  question  was  re-worded  to  read  "People  have  Medicare  "  instead 

of  "People  are  eligible  for  Medicare  "  would  that  mean  the  same  thing  to  you? 

Why/why  not?  Which  do  you  prefer?  Why? 

3.  Medicare  has  two  parts.  Part  A  covers  hospital  stays.  Part  B  covers 
medical  services  like  doctors'  visits.  Would  you  say  this  is... 

a.  True   □ , 

b.  False   Q2 

c.  Don't  know  

4.  Most  people  with  Medicare  have  to  pay  a  monthly  payment,  called  a 
premium,  for  doctors'  visits  and  other  medical  services.  Would  you  say 
this  is... 

a.  True   □  j 

b.  False   Q2 

c.  Don't  know...   Q.j 

5.  The  monthly  payment,  called  a  premium,  can  change  at  any  time  during 
the  year.  Would  you  say  this  is. . . 

a.  True   □ , 

b.  False   Q2 

c.  Don't  know   Q  j 
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6.  The  monthly  payment,  called  a  premium,  can  change  from  year  to  year. 
Would  you  say  this  is. . . 

a.  True   [Jl 

b.  False   □  2 

c.  Don't  know   □_, 

Probe:  What  does  "change  from  year  to  year"  mean  to  you  in  this  context? 

7.  Some  people  with  lower  incomes  can  get  help  paying  for  Medicare's 
premiums.  Would  you  say  this  is. . . 

a.  True   □, 

b.  False   rj2 

c.  Don't  know   □_, 

8.  Which  one  of  the  following  statements  is  true  about  what  Medicare  pays 
for?  It  pays  for. . . 

a.  All  health  care  costs   □  j 

b.  Some  health  care  costs   □  2 

c.  Only  emergency  health  care  costs   Q  3 

d.  Don't  know   fj.j 

9.  Medicare  covers  some  preventive  screening  services  such  as  flu  shots 
and  mammograms,  which  are  tests  for  breast  cancer.  Would  you  say  this 
is... 

a.  True   rj  j 

b.  False   □  2 

c.  Don't  know  

10.  Which  of  the  following  best  describes  the  Original  Medicare  plan? 
Would  you  say  Original  Medicare  is... 

a.  Medicare  Part  A,  hospital  insurance,  only   □  x 

b.  A  Medicare  option  that  is  no  longer  available   fj2 

c.  A  Medicare  option  that  has  been  available  since  Medicare  started 

and  is  still  available   d  3 

d.  Don't  know   fj.j 

11.  The  Original  Medicare  plan  is  no  longer  available  to  people  with 
Medicare.  Would  you  say  this  is. . . 

a.  True   □  j 

b.  False   Q2 

c.  Don't  know   Q.j 
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Probe:  What  does  the  term  "Original  Medicare"  mean  to  you?  Have  you 
heard  the  term  before?  [INTERVIEWER:  Record  direct  quotes  from  the 
respondent.]. 

12.      I  am  going  to  read  you  a  list  of  services.  For  each  service,  please  say 
whether  or  not  you  think  it  is  usually  covered  by  the  Medicare  Basic 
Insurance  Plan. 


Yes 

No 

DK 

a. 

Routine  eye  exams  

□  2 

□  l 

b. 

Annual  dental  exams  

••  Di 

□  2 

□  -1 

c. 

Prescription  drugs  outside  the  hospital  

••  Di 

□  2 

□  -1 

d. 

Long-term  care  in  a  nursing  home  (for  things  like  bathing 
and  dressing)  

□  i 

□  2 

□  -1 

Probe:  What  does  the  term  "Medicare  Basic  Insurance  Plan"  mean  to  you?  [INTERVIEWER: 
Record  direct  quotes  from  the  respondent] . 


III.     Medicare  +  Choice 

13.  If  a  person  thinks  that  he  is  in  a  life-threatening  situation  and  goes  to  the 
hospital  emergency  room,  will  this  visit  be  covered  if  he  has  the 
Medicare  Basic  Insurance  Plan? 

a.  Yes   □! 

b.  No   Ui 

c.  Don't  know  

14.  If  a  person  thinks  that  he  is  in  a  life-threatening  situation  and  goes  to  the 
hospital  emergency  room,  will  this  visit  be  covered  if  he  has  a  Medicare 
HMO? 

a.  Yes  □  j 

b.  No   n2 

c.  Don't  know  

Probe:  Please  think  about  the  last  two  questions  I  asked  you.  Now  I'm  going  to 
read  you  an  alternate  question  that  might  be  used  instead  of  the  two  you  just 
answered.  [INTERVIEWER:  Read  question  slowly] 
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15.      When  a  person  thinks  he  is  in  a  life-threatening  situation,  he  can  go  to 
the  hospital  emergency  room  and  the  visit  will  be  covered.  Does  this 
best  describe... 

a.  The  Medicare  Basic  Insurance  Plan   Q  j 

b.  A  Medicare  HMO   \J2 

c.  Both   □  3 

d.  Neither 

e.  Don't  know  


Probe:  Please  think  "aloud"  as  you  answer  this  question.  How  did  you  choose  your 
answer?  What  did  you  think  about? 

Please  think  about  all  three  questions.  [INTERVIEWER:  You  may  need  to  re-read  the 
questions].  What  is  the  difference  between  questions  12,  13,  and  14? 
[INTERVIEWER:  Make  sure  the  respondent  understands  that  you  are  asking  about  12 
and  13  as  a  group  compared  to  question  14]. 

Probe:  Which  do  you  prefer?  Why? 

[INTERVIEWER:  Also  probe  to  determine  whether  respondents  have  trouble 
with  the  answer  choice  "both"  in  Q14.] 

16.      Which  health  insurance  option  usually  covers  non-urgent  health  care 

services  that  people  with  Medicare  get  while  traveling  outside  the  United 


States?  Would  you  say. . . 

a.  . .  The  Medicare  Basic  Insurance  Plan   □  j 

b.  A  Medicare  HMO   Q2 

c.  Both   Q3 

d.  Neither   Q4 

e.  Don't  know   Q.j 


Probe:  Please  think  "aloud"  as  you  answer  this  question.  How  did  you  choose  your 
answer?  What  did  you  have  to  think  about? 

[INTERVIEWER:  Probe  to  determine  whether  respondents  have  difficulty  with 
the  answer  choices  "both"  and  "neither. "  Are  they  confusing  to  respondents?] 

17.      Which  of  the  following  best  describes  Medicare  managed  care?  Would 


you  say  Medicare  managed  care  is. . .. 

a.  An  HMO-type  health  plan  □  1 

b.  A  fee-for-service  health  plan  □  2 

c.  A  Medigap  supplemental  insurance  policy  □  3 

d.  Don't  know  □  -1 


Probe:  What  does  the  term  "Medicare  Managed  Care  "  mean  to  you  ?  Have 
you  heard  the  term  before?  [INTERVIEWER:  Record  direct  quotes  from  the 
respondent.]. 
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18.       Medicare+Choice  plans  are  private  Medicare  health  insurance  options 
that  people  can  sign  up  for  instead  of  the  Medicare  Basic  Insurance  Plan. 
Would  you  say  this  is. . . 

a.  True   □ ! 

b.  False   Q2 

c.  Don't  know  


Probe:  What  does  the  term  "Medicare+Choice"  mean  to  you?  Have  you 
heard  the  term  before?  [INTERVIEWER:  Record  direct  quotes  from  the 
respondent.]. 

19.  With  a  Medicare  HMO,  you  can  go  to  any  doctor  or  hospital  in  the 
United  States  and  the  visit  will  be  covered.  Would  you  say  this  is. . . 

a.  True  

b.  False  

c.  Don't  know.  

Probe:  What  does  the  term  "Medicare  HMO  "  mean  to  you  ?  Have  you  heard  the  term 
before?  [INTERVIEWER:  Record  direct  quotes  from  the  respondent.]. 

[INTERVIEWER:  Talk  with  respondents  about  the  terms  used  in  Q17,  QI8,  and 
QI9  (Medicare  Managed  Care,  Medicare+Choice,  and  Medicare  HMO).  How 
do  they  differ,  etc.  Record  verbatim  comments.] 

20.  With  a  Medicare  HMO,  you  usually  need  to  get  a  referral  to  see  a 
specialist  for  the  visit  to  be  covered.  Would  you  say  this  is. . . 

a.  True  

b.  False  

c.  Don't  know  

21 .  When  can  a  doctor  leave  his  job  at  a  Medicare  HMO? 

a.  Anytime  during  the  year  

b.  Only  in  January  

c.  Never  

d.  Don't  know  

Probe:  What  does  "leave  his  job"  mean  to  you  in  this  context? 

22.  Once  you  join  a  Medicare  HMO,  when  can  the  plan  change  your 
monthly  premium?  Would  you  say  they  can  change  it. . . 


a.  Anytime  during  the  year   Q  j 

b.  Only  in  January   Q  2 

c.  Never   Q3 

d.  Don't  know   q_j 


□  i 

□  2 

□  -1 


□  1 

□  2 

□  -1 


□  1 

□  2 

□  3 

□  -1 
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23. 


If  your  Medicare  HMO  stops  serving  people  with  Medicare  in  your  area, 
can  you  join  another  HMO  if  one  is  available? 

a.  Yes  


□  i 

□  2 

□  -1 


b 


No 


c 


Don't  know 


Probe:  In  your  own  words,  what  do  you  think  this  question  is  asking? 
Probe:  What  does  "stop  serving  people..."  mean  to  you  in  this  context? 

24.  If  your  Medicare  HMO  leaves  the  Medicare  program  and  you  do  not 
choose  another  one,  you  will  be  covered  by  the  Medicare  Basic 
Insurance  Plan.  Would  you  say  this  is. . . 

a.  True   Q  j 

b.  False   rj2 

c.  Don't  know  

Probe:  What  does  "covered  by  the  Medicare  Basic  Insurance  Plan"  mean? 
[INTERVIEWER:  Ask  the  respondent  to  describe  the  situation  presented  in 
question  24  in  their  own  words.  What  does  it  mean  ?] 

25.  If  your  Medicare  HMO  leaves  the  Medicare  program  and  you  do  not 
choose  another  one,  you  will  be  automatically  assigned  to  another  HMO. 
Would  you  say  this  is... 

a.  True   □  j 

b.  False   □  2 

c.  Don't  know  

Probe:  What  does  "automatically  assigned  to  another  HMO  "  mean  to  you  in  this 
context? 

[INTERVIEWER:  Ask  the  respondent  to  describe  the  situation  presented  in 
question  25  in  their  own  words.  What  does  it  mean  ?] 

26.  Medicare  Private  Fee-for-Service  plans  are  only  available  to  people  who 
have  health  insurance  through  their  employer.  Would  you  say  this  is. . . 

a.  True   □  j 

b.  False  

c.  Don't  know   rj_j 

27.  People  with  Medicare  can  join  a  Private  Fee-for-Service  plan  if  one  is 
available  in  their  area.  Would  you  say  this  is. . . 

a.  True   Qj 

b.  False   rj2 

c.  Don't  know   □ 
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28.  If  you  join  a  Private  Fee-for-Service  plan,  you  may  get  extra  benefits 
Medicare  doesn't  cover,  but  you  may  pay  more  for  benefits  Medicare 
does  cover.  Would  you  say  this  is... 

a.  True   □  j 

b.  False   Q2 

c.  Don't  know   Q., 


Probe:  What  does  the  term  "Private  Fee-for-Service  plan"  mean  to  you?  Have  you  heard  the  term 


before?  [INTERVIEWER:  Record  direct  quotes  from  the  respondent.]. 
Probe:  What  kind  of  "extra  benefits"  are  you  thinking  about? 

IV.      Plan  Choices  and  Health  Plan  Decision-Making 

29.  People  can  receive  their  regular  Medicare  benefits  with. . . 

a.  The  Medicare  Basic  Insurance  Plan   □  l 

b.  A  Medicare  HMO  plan   fj2 

c.  Either   □  3 

d.  Neither   Q4 

e.  Don't  know   Q.j 

Probe:  What  do  "regular  Medicare  benefits"  mean  to  you  in  this  question?  Probe: 
How  did  you  choose  your  answer?  What  did  you  have  to  think  about? 

30.  No  matter  which  Medicare  health  insurance  option  you  choose,  your  out- 
of-pocket  costs  will  be  the  same.  Would  you  say  this  is. . . 

a.  True   fj  x 

b.  False   Q2 

c.  Don't  know   Q.j 

31.  Which  type  of  Medicare  health  insurance  option  gives  you  more  freedom 
to  choose  the  doctors  and  hospitals  you  want  to  go  to?  Would  you  say. . . 

a.  The  Medicare  Basic  Insurance  Plan   □ , 

b.  A  Medicare  HMO   rj2 

c.  They  are  both  about  the  same   Q3 

d.  Don't  know  
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[INTERVIEWER:  This  is  a  VERY  important  research  question.  Have  a  full 
conversation  with  the  participant  about  it  and  ask  additional  probes  if  necessary 
and  record  verbatim  responses.  We  are  trying  to  get  at  the  best  way  to  say  that 
there  are  different  ways  that  beneficiaries  can  receive  their  Medicare  benefits 
and  that  they  differ  on  certain  features,  like  choice  of  doctors]. 
Probe:  In  your  own  words,  what  do  you  think  this  question  is  asking? 

Probe:  How  did  you  choose  your  answer?  What  did  you  have  to  think  about? 

32.      Which  type  of  Medicare  health  insurance  option  usually  has  lower  out- 
of-pocket  costs?  Would  you  say. . . 


a.  The  Basic  Medicare  Insurance  Plan   □  j 

b.  A  Medicare  HMO   □  2 

c.  They  are  both  about  the  same   \~]  3 

d.  Don't  know  

33.      Which  type  of  Medicare  health  insurance  option  is  more  likely  to  cover 
prescription  drugs  outside  of  a  hospital?  Would  you  say... 

a.  The  Basic  Medicare  Insurance  Plan   □  x 

b.  A  Medicare  HMO   [12 

c.  They  are  both  about  the  same   Q3 

d.  Don't  know   Q.j 


34.  The  Medicare  Basic  Insurance  Plan  now  covers  more  preventive  health 
care  services  like  diabetes  monitoring.  Would  you  say  that  these  same 
services  are  covered  no  matter  which  Medicare  health  insurance  option 


you  choose? 

a.  Yes   □ ! 

b.  No  •:   rj2 

c.  Don't  know   Q.j 


[INTERVIEWER:  There  are  several  questions  that  use  either  "you"  or 
"people".  Probe  to  find  out  which  the  respondent  prefers.  If  necessary,  probe 
additional  applicable  questions.] 

35.      Which  of  the  following  people  have  to  change  Medicare  health  insurance 
plans  this  year? 


a.  Everyone  who  joined  Medicare  last  year   □  l 

b.  Everyone  who  has  a  Medicare  HMO   [~]  2 

C.    People  whose  Medicare  HMOs  stop  serving  people  with  Medicare   [~]  3 

d.  Don't  know   Q.j 

[INTERVIEWER:  Note  if  you  think  the  respondent  chose  answer  "c"  because  it 
was  the  last  response  available.] 
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36.  If  you  have  a  Medicare  HMO,  you  must  also  buy  a  Medigap 
supplemental  insurance  policy.  Would  you  say  this  is... 

a.  True   Q, 

b.  False   Q2 

c.  Don't  know  

Probe:  What  does  the  term  "Medigap  supplemental  insurance  policy"  mean  to  you?  Have  you 
heard  the  term  before?  [INTERVIEWER:  Record  direct  quotes  from  the  respondent.] 

V.       Information  and  Assistance,  Beneficiary  Rights,  and  Quality  of  Care 

37.  Has  the  Medicare  program  recently  begun  to  offer  more  information  to 
help  answer  questions  about  the  Medicare  program? 

a.  Yes   □  j 

b.  No   rj2 

c.  Don't  know   Q.j 

38.  No  matter  which  Medicare  health  insurance  option  you  choose,  you  get 
the  same  quality  of  health  care.  Would  you  say  this  is. . . 

a.  True   Q  l 

b.  False   Q2 

c.  Don't  know   Q.j 

[INTERVIEWER:  Now  I'm  going  to  read  you  a  similar  version  of  the  previous 
question.] 

39.  Good  quality  health  care  means  doing  the  right  thing,  at  the  right  time,  in 
the  right  way,  and  getting  the  best  possible  results.  All  Medicare  health 
insurance  options  give  the  same  quality  of  care.  Would  you  say  this  is... 

a.  True   Q  j 

b.  False   Q2 

c.  Don't  know  

Probe:  Is  there  a  difference  between  these  two  questions?  If  yes,  what?  Which  one  is 
easier  for  you  to  understand?  Why? 

Probe:  The  second  version  of  this  question  defines  quality  as  "safe;  effective, 
and  responsive".  Did  including  the  definition  affect  your  answer?  How? 
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40.      You  have  the  right  to  ask  a  Medicare  health  insurance  plan  to  reconsider 
any  decision  it  makes  about  what  health  care  services  it  will  pay  for.  Is 
this  true  for. . . 

a.  The  Medicare  Basic  Insurance  Plan   Q  l 

b.  Medicare  HMOs   Q2 

c.  Both   Q3 

d.  Neither   Q4 

e.  Don't  know   Q.j 

Probe:  What  does  "reconsider"  mean  to  you  in  this  question? 
[INTERVIEWER:  Note  how  this  question  ad  the  answer  choices  "both"  and 
"neither"  work]. 


41.  If  you  have  Medicare,  your  health  insurance  plan  or  doctor  can  share 
your  health  information  without  your  permission.  Would  you  say  this 
is... 

a.  True   □  j 

b.  False   □  2 

c.  Don't  know   Q., 

Probe:  This  question  asks  about  sharing  your  health  information.  Who  do  you 
think  your  doctor  or  health  plan  would  share  your  health  information  with 

42.  People  with  Medicare  have  the  right  to  participate  in  their  treatment 
decisions.  Would  you  say  this  is. . . 

a.  True   □ , 

b.  False   Q2 

c.  Don't  know  

43.  Is  information  about  the  quality  of  care  you  get  with  different  Medicare 
health  insurance  options  available?... 

a-   Yes   □! 

b.  No   rj2 

c.  Don't  know  

Probe:  What  kind  of  information  are  you  thinking  about? 

44.  Does  the  Medicare  program  have  its  own  Internet  web  site? 

a.  Yes   □! 

b.  no   rj2 

c.  Don't  know  
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45.  When  can  you  call  1-800-MEDICARE  to  speak  to  someone  at  Medicare 
about  your  questions? 

a.  Only  during  business  hours   □  l 

b.  Only  on  the  weekends   □  2 

c.  24  hours  a  day/7  days  a  week   Q  3 

d.  Don't  know   Q.j 

46.  Is  there  a  service  or  a  place  in  your  area  where  people  with  Medicare  can 
get  help  understanding  and  comparing  health  insurance  options? 

a.  Yes   □  j 

b.  No   []2 

c.  Don't  know  ,  

Probe:  [If  yes]  What  kind  of  "service"  or  "place"  are  you  thinking  about? 

47.  How  much  of  the  Medicare  &  You  handbook  did  you  read?  Would  you 
say  you... 

a.  Never  really  looked  at  it  

b.  Read  parts  of  it   Q2 

c.  Read  most  or  all  of  it   [~]  3 

d.  Didn't  receive  it  or  don't  remember  receiving  it  (Skip  Next  Question)   [J 4 

e.  Don't  know  

[INTERVIEWER:  Note  whether  the  respondent  had  any  problems  with  this 
question?] 

48.  Overall,  how  useful  was  the  information  in  the  Medicare  &  You 
handbook?  Would  you  say  it  was. . . 

a.  Very  useful   □  j 

b.  Somewhat  useful   Q2 

c.  Not  very  useful   Q3 

d.  Not  at  all  useful   Q4 

e.  Don't  know   Q  j 

VI.      Medigap/Employer-Sponsored  Supplemental  Insurance 

49.  People  with  Medicare  can  buy  a  Medigap  supplemental  health  insurance 
policy  at  any  time,  regardless  of  their  health.  Would  you  say  this  is... 

a.  True   □  j 

b.  False   □  2 

c.  Don't  know   \jA 
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[INTERVIEWER:  If  low  on  time  don't  ask  Q49-Q52] 

50.  People  with  Medicare  who  also  have  health  insurance  from  an  employer 
may  choose  whether  Medicare  or  the  employer  will  be  their  primary 
insurer.  Would  you  say  this  is. . . 

a.  True   \~\] 

b.  False   Q2 

c.  Don't  know  

51.  If  you  had  a  Medigap  supplemental  insurance  policy  and  dropped  it, 
when  could  you  get  it  back? 

a.  At  any  time   □  j 

b.  Only  under  certain  conditions   Q  2 

c.  Never   Q3 

d.  Don't  know   Q.j 

52.  Insurance  companies  must  be  willing  to  sell  you  the  Medigap 
supplemental  policy  of  your  choice  for  six  months  after  you  enroll  in 
Medicare  Part  B.  Would  you  say  this  is. . . 

a.  True   □  x 

b.  False   Q2 

c.  Don't  know   Q.j 

VII.    Health  Literacy 

53.  An  appeal  is... 

a.  A  special  kind  of  complaint  you  make  if  you  disagree  with  a  □  j 
treatment  or  payment  decision  

b.  A  request  for  more  time  to  pay  your  medical  bills   Q2 

c.  Don't  know   □„! 

[Interviewer:  Note  anything  in  this  section  that  respondents  find  confusing.] 

54.  Doctors  who  take  assignment. . . 

a.  Bill  you  for  the  difference  between  what  they  charge  and  what  □  x 
Medicare  pays  for  

b.  Accept  the  amount  Medicare  pays  as  payment  in  full   Q2 

c.  Don't  know  
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55.  A  formulary  is... 

a.  A  list  of  approved  drugs   □ ; 

b.  Another  word  for  a  pharmacy   □  2 

c.  Don't  know  

The  next  set  of  questions  ask  about  terms  that  may  be  used  in  discussions  about 
Medicare.  By  answering  these  questions  you  will  help  us  identify  words  that 
might  be  confusing  or  difficult  to  understand 

56.  Preventive  care  is. . . 

a.  Care  you  get  to  keep  you  healthy   □  7 

b.  Care  you  get  to  treat  an  illness  or  disease  that  you  have   □  2 

c.  Don't  know   Q.j 

57.  A  Medicare  beneficiary  is  a. . . 

a.  Doctor  who  treats  people  with  Medicare   □  x 

b.  Person  who  gets  their  health  care  coverage  through  the  Medicare  □  2 
program  

c.  Don't  know  

58.  A  generic  drug  . . . 

a.  Does  not  work  as  well  as  a  brand  name  drug   □  j 

b.  Contains  the  same  ingredients  as  a  brand  name  drug   Q2 

c.  Don't  know  

59.  Outpatient  care  refers  to  care  you  get. . . 

a.  While  staying  overnight  at  the  hospital   □  x 

b.  Without  staying  overnight  at  the  hospital   Q2 

c.  Don't  know  

60.  A  provider  network  is  a  group  of  doctors,  hospitals  and  other  health  care 
professionals  who. . . 

a.  Work  with  an  HMO  to  take  care  of  its  members   □ ; 

b.  Are  part  of  a  labor  union   □  2 

c.  Don't  know   □_! 
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61.  A  primary  care  doctor  is  trained  to. . . 

a.  Take  care  of  your  basic  health  care  needs   Q ; 

b.  Treat  special  diseases  or  illnesses   Q2 

c.  Don't  know  

62.  A  deductible  is... 

a.  The  amount  that  Medicare  pays  for  your  health  care   □  l 

b.  The  amount  you  must  pay  for  your  health  care  before  Medicare  □  2 
begins  to  pay  

c.  Don't  know  

63.  Medigap  refers  to  private  supplemental  health  insurance  policies  that . . . 

a.  Require  you  to  go  to  doctors  or  hospitals  that  are  on  an  approved  list ...  □  x 

b.  Cover  services  that  the  Medicare  Basic  Insurance  Plan  does  not   □  2 

c.  Don't  know  
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I.        Self-reported  knowledge  questions 

1 .  How  would  you  rate  your  understanding  of  the  Medicare  program?  Would 
you  say  it  is... 

a.  Excellent   \~\  l 

b.  Very  good   Q2 

c.  Good   Q3 

d.  Fair   \J4 

e.  Poor   rj5 

2.  How  much  do  you  know  about  the  Medicare  program?  Would  you  say  you 
know... 

a.  Just  about  everything  you  need  to  know   Q ! 

b.  Most  of  what  you  need  to  know   □  2 

c.  A  little  of  what  you  need  to  know   fj  3 

d.  Almost  none  of  what  you  need  to  know   Q  4 

3.  How  easy  or  difficult  is  the  Medicare  program  to  understand?  Would  you 
say  it  is... 

a.  Very  easy  to  understand   □  j 

b.  Somewhat  easy  to  understand   rj  2 

c.  Somewhat  difficult  to  understand   Q  3 

d.  Very  difficult  to  understand   fJ34 

II.      Eligibility  for  and  Structure  of  Original  Medicare 

4.  People  are  eligible  for  Medicare  because  they  have  low  or  moderate 
incomes.  Would  you  say  this  is. . . 

a.  True   □  j 

b.  False   Q2 

c.  Don't  know   rj.j 

5.  Medicare  has  two  parts.  Part  A  covers  hospital  stays.  Part  B  covers  medical 
services  like  doctors'  visits.  Would  you  say  this  is. . . 

a.  True   □ , 

b.  False   fj2 

c.  Don't  know   □_, 
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6.  Most  people  with  Medicare  have  to  pay  a  monthly  payment,  called  a 
premium,  for  doctors'  visits  and  other  medical  services.  Would  you  say  this 
is... 

a.  True   □ , 

b.  False  Q  2 

c.  Don't  know 

7.  The  monthly  payment,  called  a  premium,  can  change  at  any  time  during  the 
year.  Would  you  say  this  is. . . 

a.  True   Q, 

b.  False   □  2 

c.  Don't  know   □_! 

8.  Some  people  with  lower  incomes  can  get  help  paying  for  Medicare's 
premiums.  Would  you  say  this  is. . . 

a.  True   rj  j 

b.  False   □  2 

c.  Don't  know  

9.  Which  one  of  the  following  is  true  about  what  Medicare  pays  for?  It  pays 
for... 

a.  All  health  care  costs   Q  l 

b.  Some  health  care  costs   □  2 

c.  Only  emergency  health  care  costs   □  3 

d.  Don't  know   fj.j 

10.  Medicare  covers  some  preventive  and  screening  services  such  as  flu  shots 
and  tests  for  breast  cancer.  Would  you  say  this  is. . . 

a.  True   □ , 

b.  False   □  2 

c.  Don't  know  

1 1 .  The  Original  Medicare  plan  is  no  longer  available.  Would  you  say  this  is. . . 

a.  True   □ , 

b.  False   fj2 

c.  Don't  know  
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12. 1  am  going  to  read  you  a  list  of  services.  For  each  one,  please  say  whether 
or  not  you  think  it  is  usually  covered  by  the  Original  Medicare  plan. 

Not  Don't 
Covered     Covered  Know 


a.  Routine  eye  exams   □  j  □  2  □  t 

b.  Annual  dental  exams   □)  [J2  D-i 

c.  Home  health  care  services....   □!  Q2  Q.j 

d.  Prescription  drugs  outside  the  hospital   Qj  Q2  □ 

e.  A  six-month  stay  in  a  nursing  home   □  j  [J2  Q.j 


13.  Imagine  that  a  person  is  in  a  life-threatening  situation  and  goes  to  the 
hospital  emergency  room,  will  this  visit  be  covered  if  the  person  has  the 
Original  Medicare  plan? 

a.  Yes   □ ! 

b.  No  :   \J2 

c.  Don't  know  


III.     Medicare  +  Choice 


14.  Imagine  that  a  person  is  in  a  life-threatening  situation  and  goes  to  the 

hospital  emergency  room,  will  this  visit  be  covered  if  the  person  belongs  to 


a  Medicare  HMO? 

a.  Yes  □  j 

b.  No   Q2 

c.  Don't  know  

15.  Which  of  the  following  best  describes  Medicare  managed  care?  Would 
you  say  Medicare  managed  care  is. . . . 

a.  An  HMO-type  health  plan   □ ! 

b.  A  fee-for-service  type  health  plan   Q  2 

c.  Both   rj3 

d.  Neither   Q4 

e.  Don't  know  

16.  People  can  sign  up  for  a  Medicare+Choice  plan  instead  of  the  Original 
Medicare  plan.  Would  you  say  this  is. . . 

a.  True   [~]  j 

b.  False   □  2 

c.  Don't  know  
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17.  With  a  Medicare  HMO,  people  can  go  to  any  doctor  or  hospital  in  the 
United  States  for  routine  care  and  the  visit  will  be  covered.  Would  you  say 
this  is... 

a.  True  

b.  False   Q2 

c.  Don't  know  

18.  With  a  Medicare  HMO,  people  almost  always  can  see  a  specialist  without  a 
referral  and  the  visit  will  be  covered.  Would  you  say  this  is.. . 

a.  True   Q1 

b.  False   □  2 

c.  Don't  know   Q.j 

19.  Once  you  join  a  Medicare  HMO,  when  can  the  plan  change  your  monthly 
premium?  Would  you  say  they  can  change  it. . . 

a.  Anytime  during  the  year   Q  j 

b.  Only  in  January   Q2 

c.  Never   Q3 

d.  Don't  know   \JA 

20.  If  your  Medicare  HMO  stops  serving  people  with  Medicare  in  your  area, 
can  you  join  another  HMO  if  one  is  available? 

a.  Yes   □  j 

b.  No   rj2 

c.  Don't  know   Q.] 

21.  If  your  Medicare  HMO  leaves  the  Medicare  program  and  you  do  not 
choose  another  one,  you  will  be  covered  by  the  Original  Medicare  plan. 
Would  you  say  this  is. . . 

a.  True   |~]  l 

b.  False   □  2 

c.  Don't  know   □_, 

22.  If  your  Medicare  HMO  leaves  the  Medicare  program  and  you  do  not 
choose  another  one,  you  will  be  assigned  to  another  HMO.  Would  you  say 
this  is... 

a.  True   □  j 

b.  False   □  2 

c.  Don't  know  
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23.  Medicare  Private  Fee-for-Service  plans  are  only  available  to  people  who 
have  health  insurance  through  their  employer.  Would  you  say  this  is. . . 

a.  True   [~j  j 

b.  False   Q2 

c.  Don't  know   Q.j 

24.  People  with  Medicare  can  join  a  Private  Fee-for-Service  plan  if  one  is 
available  in  their  area.  Would  you  say  this  is. . . 

a.  True   □  j 

b.  False   Q2 

c.  Don't  know  

25.  If  you  join  a  Private  Fee-for-Service  plan,  you  may  get  benefits  Medicare 
doesn't  cover,  but  you  may  pay  more  for  benefits  that  Medicare  does 
cover.  Would  you  say  this  is. . . 

a.  True   □  j 

b.  False   Q2 

c.  Don't  know   Q.j 

IV.      Plan  Choices  and  Health  Plan  Decision-Making 

26.  Which  health  insurance  option  usually  covers  routine  health  care  services 
that  people  with  Medicare  get  while  traveling  outside  the  United  States? 
Would  you  say... 

a.  The  Original  Medicare  plan   Q , 

b.  A  Medicare  HMO   Q2 

c.  Both   rj3 

d.  Neither   Q  4 

e.  Don't  know   □_, 

27.  People  can  receive  their  regular  Medicare  benefits  with. . . 

a.  The  Original  Medicare  plan   □  x 

b.  A  Medicare  HMO   fj2 

c.  Either  one   □  3 

d.  Neither  one   Q4 

e.  Don't  know  
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28.  No  matter  which  Medicare  health  insurance  option  you  choose,  your  out- 
of-pocket  costs  will  be  the  same.  Would  you  say  this  is. . . 

a.  True   Q  j 

b.  False   Q2 

c.  Don't  know   Q.j 

29.  Which  type  of  Medicare  health  insurance  option  gives  people  more 
freedom  to  choose  the  doctors  and  hospitals  they  want  to  go  to?  Would 
you  say... 

a.  The  Original  Medicare  plan   □  x 

b.  A  Medicare  HMO   Q2 

c.  They  are  both  about  the  same   Q3 

d.  Don't  know  

30.  Which  type  of  Medicare  health  insurance  option  is  more  likely  to  cover 
prescription  drugs  outside  of  a  hospital?  Would  you  say... 

a.  The  Original  Medicare  Plan   Q  x 

b.  A  Medicare  HMO   Q2 

c.  They  are  both  about  the  same   Q3 

d.  Don't  know   Q.j 

31.  The  Original  Medicare  plan  covers  preventive  health  care  services.  Are 
these  same  services  covered  if  a  person  chooses  a  different  Medicare  health 
insurance  option? 

a.  Yes   □! 

b.  No   Ui 

c.  Don't  know   Q.j 

32.  If  you  have  a  Medicare  HMO,  most  likely  you  would  also  buy  a  Medigap 
supplemental  insurance  policy.  Would  you  say  this  is. . . 

a.  True   □ ! 

b.  False   □  2 

c.  Don't  know  

V.       Information  and  Assistance,  Beneficiary  Rights,  and  Quality  of  Care 

33.  No  matter  which  Medicare  health  insurance  option  you  choose,  you  get  the 
same  quality  of  health  care.  Would  you  say  this  is. . . 

a.  True  

b.  False   □  2 

c.  Don't  know  
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34.  Good  quality  health  care  means  doing  the  right  thing,  at  the  right  time,  in  the 
right  way,  and  getting  the  best  possible  results.  All  Medicare  health 
insurance  options  give  the  same  quality  of  care.  Would  you  say  this  is. . . 

a.  True   r~J , 

b.  False   Q2 

c.  Don't  know   Q  j 

35.  You  have  the  right  to  ask  a  Medicare  health  insurance  plan  to  reconsider  any 
decision  it  makes  about  what  health  care  services  it  will  pay  for.  Is  this  true 
for. . . 

a.  The  Original  Medicare  plan   Q , 

b.  Medicare  HMOs   Q2 

c.  Both   Q3 

d.  Neither   Q4 

e.  Don't  know  

36.  If  you  have  Medicare,  can  your  health  insurance  plan  or  doctor  share  your 
health  information  without  your  permission? 

a.  Yes  

b-  No   D2 

c.  Don't  know   Q.j 

37.  Is  information  about  the  quality  of  care  people  get  with  different  Medicare 
health  insurance  options  available?... 

a.  Yes   □  5 

b.  No   Ui 

c.  Don't  know   Q.j 

38.  Does  the  Medicare  program  have  its  own  Internet  web  site? 

a.   Yes   □, 

b-  No   D2 

c.   Don't  know  

39.  When  can  you  call  1-800-MEDICARE  to  speak  to  someone  at  Medicare 
about  your  questions? 

a.  Only  during  business  hours   □] 

b.  Only  on  the  weekends   □  2 

c.  24  hours  a  day/7  days  a  week   □  3 

d.  Don't  know   Q.j 
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40.  Is  there  a  service  or  a  place  in  your  area  where  people  with  Medicare  can  get 
help  understanding  and  comparing  health  insurance  options? 

a.   Yes   □! 

b-  No   Q2 

c.   Don't  know  

41.  How  much  of  the  Medicare  &  You  handbook  did  you  read?  Would  you  say 
you... 

a.  Never  really  looked  at  it   Q  x 

b.  Read  parts  of  it   □  2 

c.  Read  most  or  all  of  it.   Q3 

d.  Didn't  receive  it  or  don't  remember  receiving  it  (Skip  Next  Question)   □  4 

e.  Don't  know  

42.  Overall,  how  useful  was  the  information  in  the  Medicare  &  You  handbook? 
Would  you  say  it  was. . . 

a.  Very  useful   Qj 

b.  Somewhat  useful   □  2 

c.  Not  very  useful   Q3 

d.  Not  at  all  useful   Q4 

e.  Don't  know   □_, 

VI.     Medigap/Employer-Sponsored  Supplemental  Insurance 

43.  People  with  Medicare  can  buy  a  Medigap  supplemental  health  insurance 
policy  at  any  time,  regardless  of  their  health.  Would  you  say  this  is. . . 

a.  True   □  j 

b.  False   □  2 

c. 

d.  Don't  know   □_] 

44.  People  with  Medicare  who  also  have  health  insurance  from  an  employer 
may  choose  whether  Medicare  or  the  employer  will  be  their  primary  insurer. 
Would  you  say  this  is... 

a.  True   □  j 

b.  False   Q  2 

c.  Don't  know   □_! 
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45.  If  you  had  a  Medigap  supplemental  insurance  policy  and  dropped  it,  when 
could  you  get  it  back? 

a.  At  any  time   □, 

b.  Only  under  certain  conditions   [~]  2 

c.  Never   Q3 

d.  Don't  know   Q.j 

46.  Can  an  insurance  company  refuse  to  sell  you  the  Medigap  supplemental 
policy  of  your  choice  during  the  six  months  after  you  enroll  in  Medicare  Part 
B? 

a.  Yes  

No   Q2 

c.  Don't  know  


VII.    Health  Literacy 

Introduction  to  this  section  to  be  read  by  interviewed.  The  next  set  of  questions 
asks  about  terms  that  may  be  used  in  discussions  about  Medicare.  By  answering 


these  questions  you  will  help  us  identify  words  that  might  be  confusing  or 
difficult  to  understand. 

47.  An  appeal  is... 

a.  A  special  kind  of  complaint  you  make  if  you  disagree  with  a  Qj 
treatment  or  payment  decision  

b.  A  request  for  more  time  to  pay  your  medical  bills   Q2 

c.  Don't  know  

48.  Doctors  who  take  assignment. . . 

a.  Bill  you  for  the  difference  between  what  they  charged  and  what  □  j 
Medicare  paid  

b.  Accept  the  amount  Medicare  pays  as  payment  in  full   □  2 

c.  Don't  know   □_, 

49.  A  formulary  is... 

a.  A  list  of  approved  drugs   □  j 

b.  Another  word  for  a  pharmacy   Q2 

c.  Don't  know  
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50.  Preventive  care  is... 

a.  Care  you  get  to  keep  you  healthy   □, 

b.  Care  you  get  to  treat  an  illness  or  disease  that  you  have   Q2 

c.  Don't  know   [J  { 

51.  A  Medicare  beneficiary  is  a. . . 

a.  Doctor  who  treats  people  with  Medicare   □  j 

b.  Person  who  is  insured  through  the  Medicare  program   \J2 

c.  Don't  know   Q  j 

52.  Generic  drugs  . . . 

a.  Do  not  work  as  well  as  brand  name  drugs   □  j 

b.  Do  work  as  well  as  brand  name  drugs   □  2 

c.  Don't  know   Q_j 

53.  Outpatient  care  refers  to  care  you  get. . . 

a.  While  staying  overnight  at  the  hospital   □ } 

b.  Without  staying  overnight  at  the  hospital   □  2 

c.  Don't  know   Q  j 

54.  A  provider  network  is  a  group  of  doctors,  hospitals  and  other  health  care 
professionals  who. . . 

a.  Work  with  an  HMO  to  take  care  of  its  members   □  j 

b.  specialize  in  treating  people  with  certain  diseases   □  2 

c.  Don't  know   Q.j 

55.  A  primary  care  doctor  is  trained  to. . . 

a.  Take  care  of  your  routine  health  care  needs   Q  i 

b.  Treat  special  diseases  or  illnesses   Q2 

c.  Don't  know   □_, 

56.  The  deductible  is... 

a.  The  amount  that  Medicare  pays  for  your  health  care   □  l 

b.  The  amount  you  must  pay  for  your  health  care  before  Medicare  □  2 
begins  to  pay  

c.  Don't  know  


RTI  95 


RTI 


Appendix  D 


57.  Medigap  refers  to  private  supplemental  health  insurance  policies  that  ... 

a.  Require  you  to  go  to  doctors  or  hospitals  that  are  on  an  approved  list ...  □ , 

b.  Cover  charges  that  the  Original  Medicare  plan  does  not   □  2 

c.  Don't  know   □  , 
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Set  1  (pages  6  and7) 

INSTRUCTIONS:  Please  look  over  both  pages  6  and  7  and  answer  the  following  questions. 

58.  If  people  call  1-800-MEDICARE,  they  can  listen  to  information  in  which  of  the 
following  languages?. . . 

a.  English   □  j 

b.  Spanish   Q2 

c.  Both   r-j3 

d.  Neither   Q4 

e.  Don't  know   []_, 

59.  Imagine  that  you  call  1-800-MEDICARE  because  you  lost  your  Medicare 
card  and  need  to  get  a  new  one.  Which  number  should  you  press? 

a.  1  (one)  

b.  2  (two)   rj2 

c.  3  (three)   Q3 

d.  O(zero)   Q4 

e.  Don't  know   Q_, 

60.  Imagine  that  you  call  1-800-MEDICARE  and  want  to  speak  with  a  customer 
service  representative.  Which  number  should  you  press? 

a.  1  (one)   □ , 

b.  2  (two)   rj2 

c.  3  (three)   Q3 

d.  0(zero)   d4 

e.  Don't  know   Q  j 


RTI  97 


1 


RTI 


Appendix  D 


Set  2  (pages  38-39) 

INSTRUCTIONS:  Pages  38-39  have  a  sample  Medicare  Summary  Notice.  Please  look  them  over 
and  answer  the  following  questions. 

61.  How  much  did  the  doctor  charge  Medicare  for  this  person's  office  visit  on 


March  7,  2002? 

a.  $55.00   □, 

b.  $44.35   \J2 

c.  $0   rj3 

d.  Don't  know   []  , 

62.  How  much  did  Medicare  pay  the  doctor  for  the  office  visit  on  March  7, 
2002? 

a.  $55.00   □! 

b.  $44.35   Q2 

c.  $0   rj3 

d.  Don't  know   r~j_, 

63.  According  to  this  notice,  how  much  of  this  person's  2002  deductible  has 
been  met? 

a.  $55.00  

b.  $44.35   p2 

c  $0   Q3 

d.  Don't  know   □_, 
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Set  3 

INSTRUCTIONS:  Please  read  the  information  below  and  answer  the  following  questions. 
Local  Medicare  Health  Plan  Information 

Starting  October  1,  2001,  comprehensive  information  about  the  Medicare  health  plans  in  your 
area  is  available  through  1-800-MEDICARE  (1-800-633-4227).  Customer  service 
representatives  are  available  24  hours  a  day,  including  weekends,  to  help  with  general  questions 
about  Medicare,  and  about  Medigap  policies,  Prescription  Drug  Assistance  Programs  and 
Medicare+Choice  plan  options  in  your  area. 

A  customer  service  representative  can  mail  you  detailed  information  about  Medicare  health  plans 
in  your  area,  including: 

•  Phone  numbers,  addresses  and  websites  of  each  local  plan 

•  Monthly  premium  charged 

•  Benefits  and  costs,  including  extra  benefits  like  prescription  drugs 

•  Plan  quality  and  member  satisfaction  ratings 

•  Disenrollment  information 

The  customer  service  representative  can  help  you  narrow  down  your  Medicare  health  plan  choices  using  a 
new  "Medicare  Personal  Plan  Finder."  This  tool  is  designed  to  help  you  focus  on  the  issues  most 
important  to  you  when  making  a  decision  about  a  health  plan  that  is  right  for  you.  The  customer  service 
representative  will  mail  your  personalized  results  from  this  tool  within  three  weeks  of  your  call. 

You  can  also  look  at  www.medicare.gov  on  the  Web  to  get  all  local  plan  information  and  use  the 


"Medicare  Personal  Plan  Finder." 

64.  The  purpose  of  the  "Medicare  Personal  Plan  Finder"  is  to  help  you.... 

a.  Answer  questions  about  Medicare  bills  □  l 

b.  Choose  a  primary  care  doctor  □  2 

c.  Choose  a  Medicare  health  insurance  plan  Q3 

d.  All  of  the  above  fj4 

e.  Don't  know 

65.  If  you  call  1-800-MEDICARE,  what  kind  of  information  can  you  get  from  a 
customer  service  representative? 

a.  Information  about  what  plans  will  pay  for   fj  l 

b.  Information  about  what  plans  will  cost   fj2 

c.  Information  about  the  quality  of  care  given  by  different  plans   fj3 

d.  All  of  the  above   Q4 

e.  Don't  know  
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66.  How  can  people  access  the  "Medicare  Personal  Plan  Finder"? 

a.  Using  the  Medicare  website  on  the  Internet   □  l 

b.  Using  the  toll-free  telephone  number  1-800-MEDICARE   □  2 

c.  Both   Q3 

d.  Neither   Q4 

e.  Don't  know  ,   Q  j 
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Set  4 

INSTRUCTIONS:  Please  read  the  paragraph  below  and  answer  the  following  questions. 

(From  p.  59  of  the  Handbook.  Passage  is  at  the  grade  9.4  level) 

Most  of  your  health  care  costs  are  covered  if  you  have  Medicare  and  you  qualify  for 
Medicaid.  Medicaid  is  a  joint  federal  and  state  program  that  helps  pay  medical  costs  for  some 
people  with  limited  incomes  and  resources.  Medicaid  programs  vary  from  state  to  state.  People 
with  Medicaid  may  get  coverage  for  nursing  home  care  and  outpatient  prescription  drugs  that  are 
not  covered  by  Medicare.  For  more  information  about  Medicaid,  call  your  state  medical 
assistance  office. 

67.  People  with  Medicaid  may  be  covered  for  which  of  the  following. . . 


a.  Prescription  drugs  

b.  Care  in  a  nursing  home   \J2 

c.  Both   Q3 

d.  Neither   Q4 

e.  Don't  know   []_, 

68.  People  with  Medicare  may  qualify  for  Medicaid  if. . . 

a.  They  have  incomes  that  are  more  than  $30,000  per  year   □  l 

b.  They  have  limited  incomes  or  resources   \J2 

c.  They  live  in  a  big  city   □  3 

d.  Don't  know   Q.j 

69.  Medicaid  programs. . . 

a.  Are  the  same  in  every  state   □  j 

b.  Differ  from  one  state  to  another   □  2 

c.  Are  only  for  people  with  disabilities   □  3 

d.  Don't  know   Q.j 
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Set  5  (page  29) 

INSTRUCTIONS:  Please  look  over  page  29  and  answer  the  following  questions  according  to 
information  in  the  table. 

70.  Which  of  the  following  plans  cover  dental  services? 

a.  The  Original  Medicare  plan   □  j 

b.  The  Original  Medicare  plan  with  Medigap  Plan  C   Q2 

c .  Medicare  +  Choice  HMO  Plan  #  1   □  3 

d.  None  of  the  above   Q4 

e.  Don't  know   Q  j 

71.  Which  of  the  following  plans  cover  routine  physical  exams? 

a.  The  Original  Medicare  plan   □  j 

b.  The  Original  Medicare  plan  with  Medigap  Plan  C   Q  2 

c.  Medicare  +  Choice  -  HMO  Plan  #1   Q3 

d.  None  of  the  above   Q4 

e.  Don't  know   Q_j 

72.  Which  of  the  following  plans  probably  has  the  lowest  out-of-pocket  costs? 

a.  The  Original  Medicare  plan   □  x 

b.  The  Original  Medicare  plan  with  Medigap  Plan  C   Q2 

c.  Medicare  +  Choice  -  HMO  Plan  #7   □  3 

d.  None  of  the  above   Q4 

e.  Don't  know   □.j 

73.  Which  of  the  following  plans  limits  your  choice  of  doctors? 

a.  The  Original  Medicare  plan   □  l 

b.  The  Original  Medicare  plan  with  Medigap  Plan  C   £J2 

c.  Medicare  +  Choice  -  HMO  Plan  #1   □  3 

d.  None  of  the  above   Q4 

e.  Don't  know   Q_j 
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VIII.   Cognitive  Abilities 

74.  How  often  is  the  following  statement  true  for  you?  "I  find  it  hard  to 
understand  what  I  read."  Would  you  say  it  is. . . 

a.  Never  true   □, 

b.  Seldom  true   □  2 

c.  Sometimes  true   Q3 

d.  Often  true   [~]4 

e.  Always  true   □  5 

f.  Don't  know   Q.j 

75.  For  most  people,  facts  that  are  interesting  are  easier  to  remember  than  facts 
that  are  not.  Would  you  say  you. . . 

a.  Strongly  agree   Q  j 

b.  Agree   Q2 

c.  Neither  agree  nor  disagree   Q3 

d.  Disagree   Q4 

e.  Strongly  disagree   [J5 

f.  Don't  know  

76. 1  am  good  at  remembering  the  content  of  news  articles  and  broadcasts. 
Would  you  say  you. . . 

a.  Strongly  agree   □  j 

b.  Agree   [J2 

c.  Neither  agree  nor  disagree   Q3 

d.  Disagree   Q4 

e.  Strongly  disagree   Q5 

f.  Don't  know   □_! 

77.  As  long  as  I  exercise  my  memory,  it  will  not  decline.  Would  you  say  you. . . 

a.  Strongly  agree   □ ! 

b.  Agree   \J2 

c.  Neither  agree  nor  disagree   Q3 

d.  Disagree   Q4 

e.  Strongly  disagree   Q5 

f.  Don't  know  
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78.  How  many  whole  books  have  you  read  for  pleasure  in  the  past  three 
months?  Do  not  include  reading  materials  other  than  books.  Would  you 
say. . . 

a.  No  books   □ ! 

b.  1-5  books   rj2 

c.  6-9  books   fj3 

d.  10-14  books   [J4 

e.  15  or  more  books  '.   rj5 

f.  Don't  know   fj.j 

79.  Not  including  books,  how  many  hours  per  week  do  you  spend  reading  other 
materials  (that  is,  magazines,  newspapers,  etc.)?  Would  you  say... 

a.  Zero  hours   □ } 

b.  1-5  hours   fj2 

c.  6-9  hours   fj3 

d.  10-14  hours   fj4 

IX.      Non-Knowledge  Items 

80.  How  often  do  you  look  for  information  about  Medicare  and  other  health 
insurance  options?  Would  you  say  you  look  for  information. . . 

a.  Frequently   □ } 

-    b.  Sometimes   fj2 

c.  Rarely   rj3 

d.  Never   [J4 

e.  Don't  know  

81.  In  the  past  12  months,  have  you  been  able  to  find  information  you  wanted 
about  Medicare  or  other  Medicare  health  insurance  options? 

a.  Yes   □ ! 

b.  No   Ui 

c.  I  haven't  tried  to  find  any  Medicare  information   □  .j 

d.  Don't  know   rj_2 
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82.  Most  of  what  you  have  learned  about  Medicare  is  from  your  own  personal 
experience,  rather  than  from  reading  or  hearing  about  it.  Would  you  say 
you... 

a.  Strongly  agree   □  1 

b.  Agree   \j2 

c.  Neither  agree  nor  disagree   Q3 

d.  Disagree   Q4 

e.  Strongly  disagree   Q5 

f.  Don't  know   Q.j 

83.  Who  makes  the  decision  about  which  Medicare  health  insurance  option  you 
will  get?  Would  you  say... 

a.  You  alone  make  the  decision   □ , 

b.  You  and  a  family  member,  friend,  or  insurance  counselor  make  the  □  2 
decision  together  

c.  Someone  else  makes  the  decision  for  you   Q3 

d.  Don't  know   □_, 

84.  In  your  opinion,  how  easy  is  it  to  get  information  about  Medicare?  Would 
you  say  it  is... 

a.  Very  hard  

b.  Somewhat  hard   Q2 

c.  Neither  hard  nor  easy   □  3 

d.  Somewhat  easy   Q4 

e.  Very  easy   Q5 

f.  Don't  know  

85.  How  much  do  you  trust  the  information  you  get  from  the  Medicare 
program?  Would  you  say  you  trust  it. . . 

a.  Not  at  all   □  l 

b.  A  little  ......   rj2 

c.  Some   Q3 

d.  A  lot   rj4 

e.  I  have  never  received  any  information   □  5 

f.  Don't  know  
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86.  Do  you  use  the  information  you  get  about  the  Medicare  program  to  make 
sure  that  the  health  insurance  you  have  is  the  best  choice  for  you? 

a.  Yes  ■.   Qj 

b.  No   D2 

c.  Don't  know   □_, 

87.  In  the  past  12  months,  have  you  changed  your  Medicare  health  insurance  in 
any  way? 

a.  Yes  

b.  No   Q2 

c.  Don't  know  

88.  In  the  past  12  months,  have  you  thought  about  changing  your  Medicare 
health  insurance  in  any  way? 

a.  Yes   □! 

b.  No   rj2 

c.  Don't  know   Q.j 

89.  How  hard  or  easy  is  it  for  you  to  understand  the  different  types  of  health 
insurance  options  available  to  people  with  Medicare? 

a.  Very  hard  

b.  Somewhat  hard   Q2 

c.  Neither  hard  nor  easy   □  3 

d.  Somewhat  easy   Q4 

e.  Very  easy   Q5 

f.  Don't  know  

90.  If  you  were  choosing  a  health  insurance  plan  today,  how  hard  or  easy  would 
it  be  to  decide  which  option  is  best  for  you?  Would  you  say  it  would  be. . . 

a.  Very  hard   □  ] 

b.  Somewhat  hard   □  2 

c.  Neither  hard  nor  easy   □  3 

d.  Somewhat  easy   Q4 

e.  Very  easy   Q5 

f.  Don't  know  
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91.  Before  you  became  eligible  for  Medicare,  were  you  ever  enrolled  in  an 
HMO  or  managed  care  plan? 

a.  Yes   □  j 

b-  No   rj2 

c.  Don't  know   Q.j 

92.  Have  you  ever  called  1-800-MEDICARE  to  get  help  with  Medicare 
questions? 

a.  Yes   □  j 

b.  No   D2 

c.  Don't  know   □_, 

93.  Have  you  ever  used  Medicare's  Internet  website  (www.medicare.gov)? 

a.  Yes   □! 

b.  no   n2 

c.  Don't  know   Q.j 
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